-- Logs begin at Sat 2022-08-06 00:38:58 UTC, end at Sat 2022-08-06 01:58:43 UTC. -- Aug 06 01:19:09.205670 dv-cinder-852009-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 06 01:19:09.214568 dv-cinder-852009-1-iscsi devstack@keystone.service[80189]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 06 01:19:09.216559 dv-cinder-852009-1-iscsi devstack@keystone.service[80189]: *** Starting uWSGI 2.0.18-debian (64bit) on [Sat Aug 6 01:19:09 2022] *** Aug 06 01:19:09.216559 dv-cinder-852009-1-iscsi devstack@keystone.service[80189]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 06 01:19:09.216559 dv-cinder-852009-1-iscsi devstack@keystone.service[80189]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 06 01:19:09.216559 dv-cinder-852009-1-iscsi devstack@keystone.service[80189]: nodename: dv-cinder-852009-1-iscsi Aug 06 01:19:09.216559 dv-cinder-852009-1-iscsi devstack@keystone.service[80189]: machine: x86_64 Aug 06 01:19:09.216559 dv-cinder-852009-1-iscsi devstack@keystone.service[80189]: clock source: unix Aug 06 01:19:09.216559 dv-cinder-852009-1-iscsi devstack@keystone.service[80189]: pcre jit disabled Aug 06 01:19:09.216559 dv-cinder-852009-1-iscsi devstack@keystone.service[80189]: detected number of CPU cores: 4 Aug 06 01:19:09.216559 dv-cinder-852009-1-iscsi devstack@keystone.service[80189]: current working directory: / Aug 06 01:19:09.216559 dv-cinder-852009-1-iscsi devstack@keystone.service[80189]: detected binary path: /usr/bin/uwsgi-core Aug 06 01:19:09.216559 dv-cinder-852009-1-iscsi devstack@keystone.service[80189]: your processes number limit is 47680 Aug 06 01:19:09.216559 dv-cinder-852009-1-iscsi devstack@keystone.service[80189]: your memory page size is 4096 bytes Aug 06 01:19:09.216559 dv-cinder-852009-1-iscsi devstack@keystone.service[80189]: detected max file descriptor number: 2048 Aug 06 01:19:09.216559 dv-cinder-852009-1-iscsi devstack@keystone.service[80189]: lock engine: pthread robust mutexes Aug 06 01:19:09.216559 dv-cinder-852009-1-iscsi devstack@keystone.service[80189]: thunder lock: enabled Aug 06 01:19:09.216559 dv-cinder-852009-1-iscsi devstack@keystone.service[80189]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 06 01:19:09.216559 dv-cinder-852009-1-iscsi devstack@keystone.service[80189]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 06 01:19:09.257759 dv-cinder-852009-1-iscsi devstack@keystone.service[80189]: Python main interpreter initialized at 0x557035c219c0 Aug 06 01:19:09.257759 dv-cinder-852009-1-iscsi devstack@keystone.service[80189]: python threads support enabled Aug 06 01:19:09.257759 dv-cinder-852009-1-iscsi devstack@keystone.service[80189]: your server socket listen backlog is limited to 100 connections Aug 06 01:19:09.257759 dv-cinder-852009-1-iscsi devstack@keystone.service[80189]: your mercy for graceful operations on workers is 90 seconds Aug 06 01:19:09.258124 dv-cinder-852009-1-iscsi devstack@keystone.service[80189]: mapped 403077 bytes (393 KB) for 2 cores Aug 06 01:19:09.258199 dv-cinder-852009-1-iscsi devstack@keystone.service[80189]: *** Operational MODE: preforking *** Aug 06 01:19:09.258459 dv-cinder-852009-1-iscsi devstack@keystone.service[80189]: *** uWSGI is running in multiple interpreter mode *** Aug 06 01:19:09.258459 dv-cinder-852009-1-iscsi devstack@keystone.service[80189]: spawned uWSGI master process (pid: 80189) Aug 06 01:19:09.258665 dv-cinder-852009-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 06 01:19:09.259752 dv-cinder-852009-1-iscsi devstack@keystone.service[80189]: spawned uWSGI worker 1 (pid: 80202, cores: 1) Aug 06 01:19:09.259752 dv-cinder-852009-1-iscsi devstack@keystone.service[80189]: spawned uWSGI worker 2 (pid: 80203, cores: 1) Aug 06 01:19:09.259752 dv-cinder-852009-1-iscsi devstack@keystone.service[80189]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 06 01:19:14.778594 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 06 01:19:14.778594 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: from cryptography.utils import int_from_bytes Aug 06 01:19:14.779383 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 06 01:19:14.779383 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: from cryptography.utils import int_from_bytes Aug 06 01:19:14.780256 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 06 01:19:14.780256 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: from cryptography.utils import int_from_bytes Aug 06 01:19:14.780700 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 06 01:19:14.780700 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: from cryptography.utils import int_from_bytes Aug 06 01:19:14.829669 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 06 01:19:14.829895 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 06 01:19:14.830084 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] command line args: [] {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 06 01:19:14.830482 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 06 01:19:14.830592 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 06 01:19:14.830667 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ================================================================================ {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 06 01:19:14.830733 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 06 01:19:14.830792 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] admin_token = **** {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:19:14.830860 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] command line args: [] {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 06 01:19:14.831082 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] config_dir = [] {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:19:14.831205 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 06 01:19:14.831308 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:19:14.831434 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ================================================================================ {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 06 01:19:14.831547 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] config_source = [] {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:19:14.831666 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] admin_token = **** {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:19:14.831785 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] debug = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:19:14.831785 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:19:14.832208 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] config_dir = [] {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:19:14.832283 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:19:14.832283 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:19:14.832397 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:19:14.832460 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:19:14.832522 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] config_source = [] {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:19:14.832723 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:19:14.832916 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] list_limit = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:19:14.832998 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] debug = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:19:14.833150 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] log_config_append = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:19:14.833292 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:19:14.833478 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] log_dir = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:19:14.833667 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] log_file = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:19:14.833909 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:19:14.834162 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:19:14.834411 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:19:14.834667 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:19:14.834849 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:19:14.834939 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:19:14.835027 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:19:14.835202 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:19:14.835290 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:19:14.835423 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:19:14.835501 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:19:14.835655 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:19:14.835778 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:19:14.835951 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:19:14.836031 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:19:14.836166 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] list_limit = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:19:14.836349 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:19:14.836512 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] log_config_append = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:19:14.836640 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:19:14.836765 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:19:14.836897 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:19:14.837029 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] log_dir = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:19:14.837029 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] log_file = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:19:14.837334 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:19:14.837468 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:19:14.837610 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:19:14.837739 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:19:14.837739 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:19:14.837974 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.26/identity {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:19:14.837974 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] publish_errors = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:19:14.838314 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:19:14.838314 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:19:14.838623 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:19:14.838754 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:19:14.838882 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:19:14.838882 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:19:14.838882 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:19:14.839260 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:19:14.839396 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:19:14.839529 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:19:14.839662 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] standard_threads = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:19:14.839662 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:19:14.839898 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:19:14.840022 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:19:14.840155 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:19:14.840288 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:19:14.840288 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] use_journal = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:19:14.840531 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:19:14.840656 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] use_json = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:19:14.840828 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:19:14.840973 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] use_stderr = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:19:14.840973 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] use_syslog = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:19:14.841300 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:19:14.841427 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:19:14.841552 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:19:14.841688 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.841821 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:19:14.841957 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.841957 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.841957 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.842384 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.26/identity {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:19:14.842567 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.842758 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] publish_errors = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:19:14.842955 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.843198 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:19:14.843399 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.843533 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:19:14.843666 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.843794 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:19:14.843919 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.843919 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.844255 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:19:14.844449 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.844620 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:19:14.844836 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.844995 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] standard_threads = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:19:14.845172 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.845313 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:19:14.846683 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.846970 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:19:14.847231 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.847428 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:19:14.847663 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.847856 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] use_journal = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:19:14.848049 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.848309 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] use_json = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:19:14.848631 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.848846 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] use_stderr = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:19:14.849126 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.849334 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] use_syslog = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:19:14.849586 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.849586 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.849954 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:19:14.850142 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.850370 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.850797 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.850992 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.851189 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.851384 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.851577 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.851577 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.851893 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.852053 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.852053 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.852305 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.852458 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.852458 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.852780 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.852906 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.853078 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.853215 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.853341 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.853476 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.853609 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.853736 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.853861 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.853993 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.854125 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.854269 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.854397 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.854559 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.854691 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.854824 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.854824 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.855070 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.855195 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.855325 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.855451 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.855575 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.855700 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.855832 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.855969 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.856101 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.856227 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.856357 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.856490 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.856622 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.856748 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.856872 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.856997 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.857169 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.857362 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.857496 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.857622 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.857622 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.857874 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.858506 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.858704 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.858911 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.859125 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.859351 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.859504 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.859504 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.859828 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.860003 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.860150 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.860150 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.860396 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.860540 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.860676 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.860803 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.860929 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.861084 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.861234 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.861361 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.861498 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.861625 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.861751 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.861886 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.862012 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.862137 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.862395 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.862538 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.862666 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.862792 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.862924 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.863051 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.863190 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.863319 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.863444 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.863575 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.863701 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.863856 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.863996 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.863996 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.864255 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.864385 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.864510 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.864636 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.864769 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.864896 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.865029 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.865239 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.865366 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.865503 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.865638 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.865777 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.865911 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.865911 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.866173 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.866300 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.866426 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.866560 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.866689 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.866815 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.866941 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.867067 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.867199 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.867337 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.867465 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.867592 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.867718 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.867853 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.867979 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.868113 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.868241 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.868367 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.868493 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.868627 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.868753 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.868887 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.869015 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.869175 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.869312 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.869448 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.869591 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.869728 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.869855 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.869982 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.870108 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.870233 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.870360 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.870493 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.870621 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.870747 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.870873 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.870999 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.871124 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.871258 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.871395 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.871521 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.871647 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.871782 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.871909 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.872043 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.872170 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.872296 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.872422 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.872547 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.872673 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.872876 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.873011 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.873190 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.873328 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.873456 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] auth.external = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.873585 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.873831 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.874575 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.874810 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.874978 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.875116 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.875243 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.875370 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] auth.password = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.875509 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.875637 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] auth.token = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.875764 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.875902 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.876029 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.876156 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.876290 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.876418 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.876545 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.876677 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.876804 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.876931 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.877087 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.877222 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.877350 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.877476 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.877602 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.877738 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] credential.caching = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.877873 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.878008 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.878141 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.878267 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.878429 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.878567 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.878696 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.878828 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.878964 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.879094 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.879229 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.879358 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.879495 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.879646 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.879787 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.879916 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.880043 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.880172 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.880299 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.880427 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.880565 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.880700 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.880829 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.880962 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.881118 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.881252 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.881400 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] auth.external = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.881530 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.881657 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.881793 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.881928 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.882058 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] federation.caching = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.882194 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.882323 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.882450 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] auth.password = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.882576 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.882711 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] auth.token = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.882840 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.882975 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.883103 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.883241 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.883375 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.883503 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.883630 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.883764 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.883893 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.884020 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.884147 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.884275 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.884402 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.884537 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.884665 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.884792 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] credential.caching = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.884932 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.885079 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.885222 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] identity.caching = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.885362 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.885492 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.885619 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.885746 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.885873 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.886000 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.886136 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.886264 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.886392 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.886518 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.886518 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.886767 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.886902 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.887030 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.887157 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.887283 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.887411 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.887537 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.887671 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.887804 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.887932 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.888066 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.888194 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.888327 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.888476 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.888620 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.888748 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.888875 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.889002 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.889243 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] federation.caching = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.889409 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.889627 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.889847 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.890019 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.890149 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.890286 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.890415 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.890543 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.890682 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.890811 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.890939 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.891078 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.891207 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.891335 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.891463 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.891590 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.891725 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.891865 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.891996 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.892124 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.892251 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.892387 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.892515 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.892651 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] identity.caching = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.892781 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.892909 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.893036 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.893222 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.893350 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.893489 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.893618 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.893746 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.893874 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.894001 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.894128 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.894273 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.894402 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.894529 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.894656 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.894784 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.894911 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.895047 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.895175 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.895302 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.895429 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.895556 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.895684 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.895820 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.895958 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.896086 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.896213 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.896341 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.896468 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.896607 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.896736 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.896736 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.896979 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.897132 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.897263 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.897403 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.897533 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.897660 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.897805 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.897935 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.898063 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.898199 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.898333 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.898462 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.898589 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.898716 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.user = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.898843 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.898979 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.899109 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.899236 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.899364 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.899501 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.899629 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.899770 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.899899 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.900027 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.900155 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.900287 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.900422 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.900550 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.900678 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.900805 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.900931 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.901112 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.901259 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.901399 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.901527 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.901655 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.901793 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.901920 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.902056 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.902192 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.902321 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.902448 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.902575 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.902702 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.902837 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.902966 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.903093 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.903230 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.903358 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.903484 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.903619 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.903747 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.903875 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.904001 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.904127 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.904255 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.904391 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.904391 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.904632 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.904758 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.904919 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.905139 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.905369 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.905591 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.905792 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.905929 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.906057 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.906184 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.906321 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.906450 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.user = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.906576 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.906703 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.906830 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.906957 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.907103 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.907233 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.907361 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.907488 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.907633 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.907841 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.907987 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.908116 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.908242 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.908369 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.908496 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.908622 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.908757 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.908885 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.909030 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] resource.caching = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.909215 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.909343 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.909472 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.909610 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.909739 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.909866 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.909993 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.910120 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.910247 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.910381 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.910512 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.910639 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.910765 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.910906 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.911034 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.911170 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.911298 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.911424 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.911552 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.911685 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] role.caching = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.911814 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.911949 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] role.driver = sql {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.912077 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.912204 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.912330 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.912456 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.912456 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.912717 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.912845 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.912972 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.913123 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.913255 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.913382 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.913521 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.913657 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.913784 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.913910 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.914038 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.914164 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.914299 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.914437 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.914565 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.914693 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.914820 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.914945 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.915148 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.915595 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.915737 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.915867 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.916002 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.916002 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.916255 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.916383 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.916520 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.916654 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] resource.caching = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.916783 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.916919 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.917093 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.917230 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.917367 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.917496 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.917623 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.917759 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.917889 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.918017 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.918144 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.918272 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.918408 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.918544 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.918673 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.918801 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.918929 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.919056 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.919183 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.919318 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] role.caching = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.919446 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.919574 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] role.driver = sql {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.919701 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.919828 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.919956 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.920100 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.920229 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.920357 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.920984 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.921194 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.921415 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.921626 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.921763 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.921891 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.922018 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] token.caching = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.922146 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.922272 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.922407 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.922546 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.922674 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.922801 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.922928 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.923055 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.923191 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.923319 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.923446 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.923573 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.923700 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.923827 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.923962 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.924091 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.924091 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.924341 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.924468 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.924595 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.924730 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.924858 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.924986 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.925178 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.925309 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.925444 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.925573 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.925700 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.925828 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.925955 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.926092 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.926230 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.926364 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.926493 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.926620 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] database.connection = **** {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.926756 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.926893 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.927022 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.927149 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.927285 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.927413 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.927540 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.927673 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.927801 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.927938 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.928075 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.928204 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.928333 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.928460 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.928587 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.928713 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.928848 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.928976 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.929128 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.929259 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.929387 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] token.caching = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.929513 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.929651 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.929789 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.929917 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.930043 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.930170 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.930297 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.930439 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.930568 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.930695 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.930822 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.930949 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.931075 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.931211 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.931339 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.931339 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 06 01:19:14.931598 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.931598 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.931598 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.931598 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.931598 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.931598 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.931598 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.931598 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.931598 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.932647 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.932647 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] database.connection = **** {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.932647 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.932647 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.932647 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.932647 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.932647 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.932647 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.932647 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.933485 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.933485 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.933485 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.933485 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.933485 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.933485 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.933485 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.933485 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.933485 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.934029 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.934029 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:19:14.934029 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 06 01:19:14.946274 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:19:14.946507 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:19:14.946733 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:19:14.948467 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:19:14.948707 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:19:14.948938 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:19:14.955795 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:19:14.957841 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:19:14.986812 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:19:14.987041 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:19:14.988604 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:19:14.988841 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:19:15.040157 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:19:15.040656 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:19:15.090689 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:19:15.090921 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:19:15.092089 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:19:15.092329 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:19:15.156397 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:19:15.156616 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:19:15.156786 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:19:15.156876 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:19:15.157025 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:19:15.157133 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:19:15.157277 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:19:15.157360 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:19:15.157497 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:19:15.157580 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:19:15.157717 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:19:15.157938 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:19:15.180223 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:19:15.180401 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:19:15.180487 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:19:15.180637 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:19:15.180718 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:19:15.180867 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:19:15.180950 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:19:15.181174 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:19:15.181256 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:19:15.181400 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:19:15.181491 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:19:15.181615 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:19:15.181694 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:19:15.181831 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:19:15.192623 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:19:15.193158 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:19:15.205785 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:19:15.206017 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:19:15.206213 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:19:15.206296 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:19:15.206438 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:19:15.206534 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:19:15.206654 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:19:15.206736 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:19:15.206866 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:19:15.206946 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:19:15.207078 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:19:15.207160 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:19:15.207293 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:19:15.207508 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:19:15.224486 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.224699 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 06 01:19:15.224946 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.225029 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.225188 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 06 01:19:15.225269 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 06 01:19:15.225754 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.225852 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.225972 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 06 01:19:15.226051 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.226240 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.226318 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.226496 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.226575 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.226752 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.227005 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.236239 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.236512 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.236777 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.236921 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.237066 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.237229 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.237498 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.237778 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.243978 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80202) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 01:19:15.244729 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80203) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 01:19:15.249364 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80202) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 01:19:15.249725 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.250012 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.250097 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80203) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 01:19:15.250291 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.250374 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.250567 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.250649 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.250822 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.250906 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.251089 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.251170 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.251351 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.251439 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.251614 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.251694 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.251884 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.251964 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.252149 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.252231 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.252417 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.252682 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.276030 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.276348 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.277538 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80202) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 01:19:15.277817 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.277899 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80203) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 01:19:15.278144 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.282467 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80202) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 01:19:15.282747 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.282833 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80203) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 01:19:15.283029 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.283111 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.283370 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.289756 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80202) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 01:19:15.290040 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80203) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 01:19:15.290123 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.290361 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.294135 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80202) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 01:19:15.294419 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.294518 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80203) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 01:19:15.294700 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.294782 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.294993 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.295074 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.295280 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.295363 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.295586 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.295671 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.295869 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.296089 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.296238 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.296392 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.296534 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.313126 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.313282 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.313410 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.313559 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.317373 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80203) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 01:19:15.317510 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80202) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 01:19:15.320534 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.320631 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.320830 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.320913 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.325303 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80203) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 01:19:15.325390 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80202) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 01:19:15.329320 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80202) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 01:19:15.329463 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80203) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 01:19:15.332471 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.332682 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.332765 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.332983 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.333084 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.333294 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.333376 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.333583 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.333677 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.333856 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.333937 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.334140 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.349934 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80202) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 01:19:15.350215 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.350398 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80203) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 01:19:15.350732 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.350943 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.351024 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.351211 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.351485 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.357881 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.358575 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.359446 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.359719 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.360129 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.360417 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.362576 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80202) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 01:19:15.362855 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.363144 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.363300 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80203) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 01:19:15.363425 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.363581 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.363703 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.363881 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.364162 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.364440 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.375664 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80202) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 01:19:15.375938 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.376214 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.376490 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.376586 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80203) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 01:19:15.376768 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.376860 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.377041 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.377156 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.377346 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.377429 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.377646 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.377922 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.378208 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.393501 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80202) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 01:19:15.394548 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80203) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 01:19:15.396500 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80202) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 01:19:15.397584 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80203) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 01:19:15.399493 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.400550 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.402115 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.403096 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.403686 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80202) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 01:19:15.403932 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.404224 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.404779 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80203) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 01:19:15.405235 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.405733 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.411081 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.412435 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.412686 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80202) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 01:19:15.414094 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80203) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 01:19:15.415652 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.415941 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.416221 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.416491 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.417067 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.417365 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.417648 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.417923 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.424697 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80202) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 01:19:15.424977 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.425286 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.426100 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80203) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 01:19:15.426390 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.426671 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.432188 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80202) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 01:19:15.432466 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.432743 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.433015 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.433310 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.433579 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.433663 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80203) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 01:19:15.433845 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.433928 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.434122 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.434207 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.434401 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.434501 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.434675 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.434756 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.434947 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.435037 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.435218 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.435310 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.435515 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.435599 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.435805 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.435885 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.436694 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.436967 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.437269 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.437532 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.437806 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:19:15.470539 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x557035c219c0 pid: 80202 (default app) Aug 06 01:19:15.471985 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x557035c219c0 pid: 80203 (default app) Aug 06 01:19:15.473893 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894ebbf7-54c2-4ac7-8f6c-bd4474db38d2 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:15.474187 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894ebbf7-54c2-4ac7-8f6c-bd4474db38d2 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:15.474463 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894ebbf7-54c2-4ac7-8f6c-bd4474db38d2 None None] PATH_INFO: `/v3` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:15.479051 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1/1] 10.222.0.26 () {50 vars in 791 bytes} [Sat Aug 6 01:19:15 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:19:23.112341 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38d08c8-38fe-420f-b820-98dca3fc180e None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:23.112903 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38d08c8-38fe-420f-b820-98dca3fc180e None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:23.112903 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38d08c8-38fe-420f-b820-98dca3fc180e None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:23.117386 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1/2] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:19:23 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:19:23.122572 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ad2e57-14b6-45c1-bc0d-ae26e4d22920 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:23.123001 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ad2e57-14b6-45c1-bc0d-ae26e4d22920 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:23.123129 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ad2e57-14b6-45c1-bc0d-ae26e4d22920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:23.192738 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: INFO dbcounter [None req-56ad2e57-14b6-45c1-bc0d-ae26e4d22920 None None] Registered counter for database keystone Aug 06 01:19:23.211897 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG oslo_db.sqlalchemy.engines [None req-56ad2e57-14b6-45c1-bc0d-ae26e4d22920 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80202) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 06 01:19:23.218830 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writer thread running {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 06 01:19:23.396430 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG passlib.handlers.bcrypt [None req-56ad2e57-14b6-45c1-bc0d-ae26e4d22920 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80202) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 06 01:19:23.396847 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG passlib.handlers.bcrypt [None req-56ad2e57-14b6-45c1-bc0d-ae26e4d22920 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80202) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 06 01:19:23.450909 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-56ad2e57-14b6-45c1-bc0d-ae26e4d22920 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:19:23.453225 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-56ad2e57-14b6-45c1-bc0d-ae26e4d22920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:19:23.582536 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 2/3] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:19:23 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 461 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 06 01:19:23.744016 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38d08c8-38fe-420f-b820-98dca3fc180e None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:23.814708 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: INFO dbcounter [None req-e38d08c8-38fe-420f-b820-98dca3fc180e None None] Registered counter for database keystone Aug 06 01:19:23.836198 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG oslo_db.sqlalchemy.engines [None req-e38d08c8-38fe-420f-b820-98dca3fc180e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80203) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 06 01:19:23.842302 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writer thread running {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 06 01:19:23.962744 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38d08c8-38fe-420f-b820-98dca3fc180e None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:23.967088 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3445d9-e0bf-4349-bf80-b285169d39d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:23.967813 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3445d9-e0bf-4349-bf80-b285169d39d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:23.968100 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3445d9-e0bf-4349-bf80-b285169d39d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:23.968380 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3445d9-e0bf-4349-bf80-b285169d39d5 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:23.976647 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3445d9-e0bf-4349-bf80-b285169d39d5 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:24.029415 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: WARNING keystone.common.rbac_enforcer.enforcer [None req-0d3445d9-e0bf-4349-bf80-b285169d39d5 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 06 01:19:24.029720 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3445d9-e0bf-4349-bf80-b285169d39d5 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:24.039343 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 2/4] 10.222.0.26 () {58 vars in 1324 bytes} [Sat Aug 6 01:19:23 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 06 01:19:24.045931 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ad2e57-14b6-45c1-bc0d-ae26e4d22920 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:24.060450 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ad2e57-14b6-45c1-bc0d-ae26e4d22920 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:24.066072 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e180c186-1fe2-453b-b07d-bf484bd587a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:24.067180 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e180c186-1fe2-453b-b07d-bf484bd587a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:24.067472 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e180c186-1fe2-453b-b07d-bf484bd587a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:24.067757 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e180c186-1fe2-453b-b07d-bf484bd587a2 admin admin] PATH_INFO: `/v3/projects/477e329599354f58af86e800a455a8b2` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:24.069278 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e180c186-1fe2-453b-b07d-bf484bd587a2 admin admin] RBAC: Authorizing `identity:get_project(project_id=477e329599354f58af86e800a455a8b2)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:24.119573 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: WARNING keystone.common.rbac_enforcer.enforcer [None req-e180c186-1fe2-453b-b07d-bf484bd587a2 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 06 01:19:24.119917 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e180c186-1fe2-453b-b07d-bf484bd587a2 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:24.125212 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 3/5] 10.222.0.26 () {56 vars in 1207 bytes} [Sat Aug 6 01:19:24 2022] GET /identity/v3/projects/477e329599354f58af86e800a455a8b2 => generated 344 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:19:26.109004 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a113c809-21c1-4433-9345-e0d40a1281e8 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:26.109586 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a113c809-21c1-4433-9345-e0d40a1281e8 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:26.109586 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a113c809-21c1-4433-9345-e0d40a1281e8 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:26.113776 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 3/6] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:19:26 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:19:26.119227 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e108ea93-1dc5-4690-991e-6e4a61d3604a None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:26.119595 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e108ea93-1dc5-4690-991e-6e4a61d3604a None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:26.119798 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e108ea93-1dc5-4690-991e-6e4a61d3604a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:26.140032 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-e108ea93-1dc5-4690-991e-6e4a61d3604a None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:19:26.141560 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-e108ea93-1dc5-4690-991e-6e4a61d3604a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:19:26.149348 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 4/7] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:19:26 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 06 01:19:26.171530 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a113c809-21c1-4433-9345-e0d40a1281e8 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:26.183644 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a113c809-21c1-4433-9345-e0d40a1281e8 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:26.188125 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552c3639-a240-43e7-9ce1-376c73d1f089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:26.188738 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552c3639-a240-43e7-9ce1-376c73d1f089 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:26.189025 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552c3639-a240-43e7-9ce1-376c73d1f089 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:26.189339 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552c3639-a240-43e7-9ce1-376c73d1f089 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:26.197106 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552c3639-a240-43e7-9ce1-376c73d1f089 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:26.198890 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552c3639-a240-43e7-9ce1-376c73d1f089 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:26.208299 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 4/8] 10.222.0.26 () {58 vars in 1324 bytes} [Sat Aug 6 01:19:26 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 06 01:19:26.213605 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e108ea93-1dc5-4690-991e-6e4a61d3604a None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:26.223460 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e108ea93-1dc5-4690-991e-6e4a61d3604a None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:26.228138 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd026de1-0da7-4e9d-a79a-28516cc2e48c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:26.228774 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd026de1-0da7-4e9d-a79a-28516cc2e48c admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:26.229083 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd026de1-0da7-4e9d-a79a-28516cc2e48c admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:26.229367 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd026de1-0da7-4e9d-a79a-28516cc2e48c admin admin] PATH_INFO: `/v3/users/ca5d3774b93a41d1a8b95796b7899d4c` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:26.230733 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd026de1-0da7-4e9d-a79a-28516cc2e48c admin admin] RBAC: Authorizing `identity:get_user(user_id=ca5d3774b93a41d1a8b95796b7899d4c)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:26.232570 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd026de1-0da7-4e9d-a79a-28516cc2e48c admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:26.234342 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 5/9] 10.222.0.26 () {56 vars in 1198 bytes} [Sat Aug 6 01:19:26 2022] GET /identity/v3/users/ca5d3774b93a41d1a8b95796b7899d4c => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:19:28.328398 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211b270c-eab9-4418-99cb-118df6b02958 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:28.328950 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211b270c-eab9-4418-99cb-118df6b02958 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:28.328950 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211b270c-eab9-4418-99cb-118df6b02958 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:28.330318 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 5/10] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:19:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 01:19:28.335566 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ada2bb-5870-49e5-8cfe-82e94a221d0c None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:28.335985 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ada2bb-5870-49e5-8cfe-82e94a221d0c None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:28.336114 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ada2bb-5870-49e5-8cfe-82e94a221d0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:28.340653 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eddab89-3b1a-4b1f-be67-33797ed49070 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:28.340941 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eddab89-3b1a-4b1f-be67-33797ed49070 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:28.341242 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eddab89-3b1a-4b1f-be67-33797ed49070 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:28.343011 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 6/11] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:19:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 01:19:28.348337 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebbddc2-0f16-4bb7-919b-ab35aa5b376c None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:28.348629 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebbddc2-0f16-4bb7-919b-ab35aa5b376c None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:28.348910 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebbddc2-0f16-4bb7-919b-ab35aa5b376c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:28.355794 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-27ada2bb-5870-49e5-8cfe-82e94a221d0c None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:19:28.356976 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-27ada2bb-5870-49e5-8cfe-82e94a221d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:19:28.391176 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG passlib.handlers.bcrypt [None req-3ebbddc2-0f16-4bb7-919b-ab35aa5b376c None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80203) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 06 01:19:28.391618 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG passlib.handlers.bcrypt [None req-3ebbddc2-0f16-4bb7-919b-ab35aa5b376c None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80203) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 06 01:19:28.438366 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 6/12] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:19:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:19:28.438548 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-3ebbddc2-0f16-4bb7-919b-ab35aa5b376c None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:19:28.440261 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-3ebbddc2-0f16-4bb7-919b-ab35aa5b376c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:19:28.460619 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ada2bb-5870-49e5-8cfe-82e94a221d0c None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:28.484862 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:28.541827 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 7/13] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:19:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 01:19:28.565006 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ada2bb-5870-49e5-8cfe-82e94a221d0c None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:28.565547 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebbddc2-0f16-4bb7-919b-ab35aa5b376c None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:28.599452 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a59a53-46a1-4952-9e59-0ca0c908c388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:28.600005 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a59a53-46a1-4952-9e59-0ca0c908c388 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:28.600289 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a59a53-46a1-4952-9e59-0ca0c908c388 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:28.600832 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a59a53-46a1-4952-9e59-0ca0c908c388 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:28.669303 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebbddc2-0f16-4bb7-919b-ab35aa5b376c None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:28.672379 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:28.675373 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a59a53-46a1-4952-9e59-0ca0c908c388 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:28.676860 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a59a53-46a1-4952-9e59-0ca0c908c388 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:28.699376 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:28.708326 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309fd05c-9018-4d2c-8de8-4164180b1490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:28.709468 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309fd05c-9018-4d2c-8de8-4164180b1490 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:28.709753 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309fd05c-9018-4d2c-8de8-4164180b1490 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:28.710032 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309fd05c-9018-4d2c-8de8-4164180b1490 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:28.722515 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309fd05c-9018-4d2c-8de8-4164180b1490 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:19:28.731315 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-309fd05c-9018-4d2c-8de8-4164180b1490 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:19:28.731315 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:19:28.731315 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:19:28.731315 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:19:28.731315 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:19:28.731315 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:19:28.731315 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:19:28.731315 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:19:28.731315 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:19:28.731315 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:19:28.731315 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:19:28.731315 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:19:28.731315 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:19:28.731315 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:19:28.731315 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:19:28.731315 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:19:28.732197 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:19:28.732197 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:19:28.732197 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:19:28.732197 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:19:28.732197 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:19:28.732197 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:19:28.732197 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:19:28.732197 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:19:28.732197 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:19:28.732197 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:19:28.732197 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:19:28.732197 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:19:28.732197 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:19:28.732197 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:19:28.732197 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:19:28.732197 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:19:28.733493 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:19:28.733493 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:19:28.733493 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:19:28.733493 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:19:28.733493 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:19:28.733493 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:19:28.733493 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:19:28.733493 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:19:28.733493 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:19:28.733493 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:19:28.733493 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:19:28.733493 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:19:28.733493 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 8/14] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Aug 6 01:19:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:19:28.738458 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309fd05c-9018-4d2c-8de8-4164180b1490 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:28.778098 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 7/15] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Aug 6 01:19:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 06 01:19:28.783715 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a59a53-46a1-4952-9e59-0ca0c908c388 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:28.839547 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309fd05c-9018-4d2c-8de8-4164180b1490 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:28.872794 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea96f32-e8e5-4b80-819d-e0251e02fd11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:28.873921 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea96f32-e8e5-4b80-819d-e0251e02fd11 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:28.874207 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea96f32-e8e5-4b80-819d-e0251e02fd11 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:28.874489 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea96f32-e8e5-4b80-819d-e0251e02fd11 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:28.875307 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea96f32-e8e5-4b80-819d-e0251e02fd11 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:19:28.875621 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea96f32-e8e5-4b80-819d-e0251e02fd11 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:28.877152 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea96f32-e8e5-4b80-819d-e0251e02fd11 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:28.880804 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a59a53-46a1-4952-9e59-0ca0c908c388 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:28.890047 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 9/16] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Aug 6 01:19:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:19:28.895619 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea96f32-e8e5-4b80-819d-e0251e02fd11 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:28.902993 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:28.915569 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c54e8d-6422-45d3-8c63-6a0d51d054d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:28.916213 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c54e8d-6422-45d3-8c63-6a0d51d054d4 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:28.916495 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c54e8d-6422-45d3-8c63-6a0d51d054d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:28.916773 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c54e8d-6422-45d3-8c63-6a0d51d054d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:28.922620 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:28.927843 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c54e8d-6422-45d3-8c63-6a0d51d054d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:28.929350 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c54e8d-6422-45d3-8c63-6a0d51d054d4 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:28.948938 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-f1c54e8d-6422-45d3-8c63-6a0d51d054d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:19:28.948938 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:19:28.948938 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:19:28.948938 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:19:28.948938 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:19:28.948938 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:19:28.948938 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:19:28.948938 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:19:28.948938 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:19:28.948938 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:19:28.948938 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:19:28.948938 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:19:28.948938 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:19:28.948938 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:19:28.948938 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:19:28.949992 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:19:28.949992 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:19:28.949992 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:19:28.949992 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:19:28.949992 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:19:28.949992 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:19:28.949992 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:19:28.949992 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:19:28.949992 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:19:28.949992 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:19:28.949992 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:19:28.949992 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:19:28.949992 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:19:28.949992 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:19:28.949992 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:19:28.949992 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:19:28.949992 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:19:28.950821 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:19:28.950821 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:19:28.950821 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:19:28.950821 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:19:28.950821 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 8/17] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:19:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:19:28.956634 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c54e8d-6422-45d3-8c63-6a0d51d054d4 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:29.002976 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea96f32-e8e5-4b80-819d-e0251e02fd11 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:29.040799 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c907ba-090f-4cac-9bfb-0b60a86a1c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:29.041606 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c907ba-090f-4cac-9bfb-0b60a86a1c06 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:29.041957 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c907ba-090f-4cac-9bfb-0b60a86a1c06 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:29.042305 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c907ba-090f-4cac-9bfb-0b60a86a1c06 None admin] PATH_INFO: `/v3/users/ca5d3774b93a41d1a8b95796b7899d4c` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:29.044007 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c907ba-090f-4cac-9bfb-0b60a86a1c06 None admin] RBAC: Authorizing `identity:get_user(user_id=ca5d3774b93a41d1a8b95796b7899d4c)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:29.045701 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c907ba-090f-4cac-9bfb-0b60a86a1c06 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:29.047518 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 10/18] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Aug 6 01:19:28 2022] GET /identity/v3/users/ca5d3774b93a41d1a8b95796b7899d4c => generated 247 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:19:29.053080 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c907ba-090f-4cac-9bfb-0b60a86a1c06 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:29.148671 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:29.157142 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c907ba-090f-4cac-9bfb-0b60a86a1c06 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:29.162460 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c54e8d-6422-45d3-8c63-6a0d51d054d4 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:29.190633 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a9faac-bcd6-4841-ab9e-c55069aabe69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:29.191515 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a9faac-bcd6-4841-ab9e-c55069aabe69 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:29.191800 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a9faac-bcd6-4841-ab9e-c55069aabe69 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:29.192080 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a9faac-bcd6-4841-ab9e-c55069aabe69 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:29.193277 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a9faac-bcd6-4841-ab9e-c55069aabe69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:29.194739 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a9faac-bcd6-4841-ab9e-c55069aabe69 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:29.195229 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c591e06-cd9b-48a7-91df-fbb50f14efce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:29.195847 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c591e06-cd9b-48a7-91df-fbb50f14efce None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:29.196133 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c591e06-cd9b-48a7-91df-fbb50f14efce None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:29.196331 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 11/19] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:19:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:19:29.196443 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c591e06-cd9b-48a7-91df-fbb50f14efce None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:29.197130 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c591e06-cd9b-48a7-91df-fbb50f14efce None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:19:29.197480 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c591e06-cd9b-48a7-91df-fbb50f14efce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:29.198813 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c591e06-cd9b-48a7-91df-fbb50f14efce None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:29.202815 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a9faac-bcd6-4841-ab9e-c55069aabe69 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:29.216815 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 9/20] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Aug 6 01:19:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:19:29.310024 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a9faac-bcd6-4841-ab9e-c55069aabe69 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:29.346040 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bb6d6d-cbc2-4387-927d-b32c8b38105c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:29.346678 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bb6d6d-cbc2-4387-927d-b32c8b38105c None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:29.346949 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bb6d6d-cbc2-4387-927d-b32c8b38105c None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:29.347228 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bb6d6d-cbc2-4387-927d-b32c8b38105c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:29.348036 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bb6d6d-cbc2-4387-927d-b32c8b38105c None admin] RBAC: Adding query filter params (role.id=4559b9a0837249f895a13c70096f8d38, scope.domain.id=default, user.id=ca5d3774b93a41d1a8b95796b7899d4c) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:19:29.348350 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bb6d6d-cbc2-4387-927d-b32c8b38105c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:29.349859 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bb6d6d-cbc2-4387-927d-b32c8b38105c None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:29.359693 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 12/21] 10.222.0.26 () {56 vars in 1322 bytes} [Sat Aug 6 01:19:29 2022] GET /identity/v3/role_assignments?user.id=ca5d3774b93a41d1a8b95796b7899d4c&scope.domain.id=default&role.id=4559b9a0837249f895a13c70096f8d38 => generated 233 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:19:31.498204 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cff9fa-fcbe-4104-883d-ab83ac48b049 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:31.499620 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cff9fa-fcbe-4104-883d-ab83ac48b049 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:31.500121 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cff9fa-fcbe-4104-883d-ab83ac48b049 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:31.501715 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 10/22] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:19:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:19:31.507269 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fcf08a-e021-4768-8a17-26783e9b183b None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:31.508137 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fcf08a-e021-4768-8a17-26783e9b183b None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:31.508633 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fcf08a-e021-4768-8a17-26783e9b183b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:31.535500 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-33fcf08a-e021-4768-8a17-26783e9b183b None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:19:31.537076 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-33fcf08a-e021-4768-8a17-26783e9b183b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:19:31.559325 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:31.622395 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 13/23] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:19:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:19:31.667149 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cff9fa-fcbe-4104-883d-ab83ac48b049 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:31.707013 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:31.727354 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cad0daa-7e8d-4827-a025-ced6bc75431c None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:31.727672 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cad0daa-7e8d-4827-a025-ced6bc75431c None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:31.727922 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cad0daa-7e8d-4827-a025-ced6bc75431c None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:31.729335 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 14/24] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:19:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:19:31.735236 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517e7419-d312-46e4-bb00-bda866a4eba2 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:31.735523 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517e7419-d312-46e4-bb00-bda866a4eba2 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:31.735800 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517e7419-d312-46e4-bb00-bda866a4eba2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:31.766228 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-517e7419-d312-46e4-bb00-bda866a4eba2 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:19:31.767433 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-517e7419-d312-46e4-bb00-bda866a4eba2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:19:31.841774 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cff9fa-fcbe-4104-883d-ab83ac48b049 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:31.892080 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 15/25] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:19:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 01:19:31.893403 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c75a24-b31e-49db-b0b8-7d0f8cf8b850 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:31.893692 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c75a24-b31e-49db-b0b8-7d0f8cf8b850 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:31.893970 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c75a24-b31e-49db-b0b8-7d0f8cf8b850 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:31.896711 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 16/26] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:19:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:19:31.912649 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac61cf3-7418-40c9-8504-9212662442c3 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:31.912934 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac61cf3-7418-40c9-8504-9212662442c3 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:31.913240 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac61cf3-7418-40c9-8504-9212662442c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:31.938374 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ab0f07-31b2-4ce0-9a51-680360eb13b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:31.939041 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ab0f07-31b2-4ce0-9a51-680360eb13b6 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:31.939326 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ab0f07-31b2-4ce0-9a51-680360eb13b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:31.939599 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ab0f07-31b2-4ce0-9a51-680360eb13b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:31.941404 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-9ac61cf3-7418-40c9-8504-9212662442c3 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:19:31.942617 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-9ac61cf3-7418-40c9-8504-9212662442c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:19:31.966199 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ab0f07-31b2-4ce0-9a51-680360eb13b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:31.968271 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ab0f07-31b2-4ce0-9a51-680360eb13b6 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:31.992743 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-13ab0f07-31b2-4ce0-9a51-680360eb13b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:19:31.992743 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:19:31.992743 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:19:31.992743 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:19:31.992743 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:19:31.992743 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:19:31.992743 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:19:31.992743 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:19:31.992743 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:19:31.992743 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:19:31.992743 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:19:31.992743 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:19:31.992743 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:19:31.992743 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:19:31.992743 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:19:31.993621 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:19:31.993621 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:19:31.993621 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:19:31.993621 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:19:31.993621 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:19:31.993621 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:19:31.993621 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:19:31.993621 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:19:31.993621 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:19:31.993621 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:19:31.993621 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:19:31.993621 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:19:31.993621 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:19:31.993621 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:19:31.993621 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:19:31.993621 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:19:31.993621 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:19:31.994844 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:19:31.994844 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:19:31.994844 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:19:31.994844 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:19:31.994844 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 11/27] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:19:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:19:31.995103 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ab0f07-31b2-4ce0-9a51-680360eb13b6 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:32.062602 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 17/28] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:19:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 01:19:32.063972 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac61cf3-7418-40c9-8504-9212662442c3 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:32.110558 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:32.142948 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:32.160579 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ab0f07-31b2-4ce0-9a51-680360eb13b6 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:32.213860 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526f756f-b70d-44ef-aa37-1aa06d87ccfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:32.214451 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526f756f-b70d-44ef-aa37-1aa06d87ccfa None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:32.214728 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526f756f-b70d-44ef-aa37-1aa06d87ccfa None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:32.215002 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526f756f-b70d-44ef-aa37-1aa06d87ccfa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:32.237748 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526f756f-b70d-44ef-aa37-1aa06d87ccfa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:19:32.246520 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-526f756f-b70d-44ef-aa37-1aa06d87ccfa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:19:32.246520 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:19:32.246520 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:19:32.246520 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:19:32.246520 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:19:32.246520 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:19:32.246520 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:19:32.246520 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:19:32.246520 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:19:32.246520 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:19:32.246520 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:19:32.246520 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:19:32.246520 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:19:32.246520 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:19:32.246520 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:19:32.246520 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:19:32.247366 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:19:32.247366 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:19:32.247366 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:19:32.247366 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:19:32.247366 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:19:32.247366 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:19:32.247366 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:19:32.247366 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:19:32.247366 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:19:32.247366 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:19:32.247366 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:19:32.247366 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:19:32.247366 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:19:32.247366 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:19:32.247366 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:19:32.247366 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:19:32.248535 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:19:32.248535 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:19:32.248535 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:19:32.248535 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:19:32.248535 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:19:32.248535 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:19:32.248535 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:19:32.248535 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:19:32.248535 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:19:32.248535 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:19:32.248535 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:19:32.248535 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:19:32.248535 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 12/29] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Aug 6 01:19:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:19:32.252047 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526f756f-b70d-44ef-aa37-1aa06d87ccfa None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:32.271969 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac61cf3-7418-40c9-8504-9212662442c3 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:32.416338 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526f756f-b70d-44ef-aa37-1aa06d87ccfa None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:32.460124 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9a2a72-08f0-4893-910c-abee4cbf06b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:32.461176 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9a2a72-08f0-4893-910c-abee4cbf06b9 None admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:32.461713 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9a2a72-08f0-4893-910c-abee4cbf06b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:32.462297 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9a2a72-08f0-4893-910c-abee4cbf06b9 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:32.463212 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9a2a72-08f0-4893-910c-abee4cbf06b9 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:32.465096 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9a2a72-08f0-4893-910c-abee4cbf06b9 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:32.470979 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab09efe-41f8-4618-8b51-53c34c22547a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:32.471611 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab09efe-41f8-4618-8b51-53c34c22547a None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:32.471897 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab09efe-41f8-4618-8b51-53c34c22547a None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:32.472178 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab09efe-41f8-4618-8b51-53c34c22547a None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:32.472888 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab09efe-41f8-4618-8b51-53c34c22547a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:19:32.473248 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab09efe-41f8-4618-8b51-53c34c22547a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:32.474834 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab09efe-41f8-4618-8b51-53c34c22547a None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:32.498974 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 18/30] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Aug 6 01:19:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 435 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:19:32.500130 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab09efe-41f8-4618-8b51-53c34c22547a None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:32.581373 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:32.606371 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 13/31] 10.222.0.26 () {60 vars in 1131 bytes} [Sat Aug 6 01:19:32 2022] POST /identity/v3/roles => generated 219 bytes in 358 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:19:32.608872 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbdf49f-b395-42d0-b30b-5e5357140033 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:32.609305 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbdf49f-b395-42d0-b30b-5e5357140033 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:32.609691 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbdf49f-b395-42d0-b30b-5e5357140033 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:32.613304 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 14/32] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:19:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:19:32.613304 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbdf49f-b395-42d0-b30b-5e5357140033 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:32.619867 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab09efe-41f8-4618-8b51-53c34c22547a None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:32.629077 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:32.680992 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eceaf395-1c1d-4ac5-8cc8-56ba1aba88c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:32.681715 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceaf395-1c1d-4ac5-8cc8-56ba1aba88c3 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:32.682073 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceaf395-1c1d-4ac5-8cc8-56ba1aba88c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:32.682359 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceaf395-1c1d-4ac5-8cc8-56ba1aba88c3 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:32.683203 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eceaf395-1c1d-4ac5-8cc8-56ba1aba88c3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:19:32.683520 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eceaf395-1c1d-4ac5-8cc8-56ba1aba88c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:32.684972 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eceaf395-1c1d-4ac5-8cc8-56ba1aba88c3 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:32.709172 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 19/33] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Aug 6 01:19:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:19:32.710468 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8439e3-7aad-4a96-ad09-933201f07d8d None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:32.710755 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8439e3-7aad-4a96-ad09-933201f07d8d None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:32.711040 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8439e3-7aad-4a96-ad09-933201f07d8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:32.753607 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-dc8439e3-7aad-4a96-ad09-933201f07d8d None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:19:32.756643 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-dc8439e3-7aad-4a96-ad09-933201f07d8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:19:32.829447 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbdf49f-b395-42d0-b30b-5e5357140033 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:32.868507 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9778124e-828e-4e74-b6ba-9ac2d1d6c63f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:32.869207 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9778124e-828e-4e74-b6ba-9ac2d1d6c63f None admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:32.869505 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9778124e-828e-4e74-b6ba-9ac2d1d6c63f None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:32.869789 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9778124e-828e-4e74-b6ba-9ac2d1d6c63f None admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:32.870500 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9778124e-828e-4e74-b6ba-9ac2d1d6c63f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:32.871083 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 20/34] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:19:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:19:32.872185 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8439e3-7aad-4a96-ad09-933201f07d8d None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:32.882696 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9778124e-828e-4e74-b6ba-9ac2d1d6c63f None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:32.930092 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 15/35] 10.222.0.26 () {60 vars in 1141 bytes} [Sat Aug 6 01:19:32 2022] POST /identity/v3/projects => generated 301 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:19:32.934670 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9778124e-828e-4e74-b6ba-9ac2d1d6c63f None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:32.949768 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:32.995369 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8439e3-7aad-4a96-ad09-933201f07d8d None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:33.035222 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef205fcc-40c4-412f-8ea2-7013ddc5c955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:33.036260 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef205fcc-40c4-412f-8ea2-7013ddc5c955 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:33.036744 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef205fcc-40c4-412f-8ea2-7013ddc5c955 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:33.037243 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef205fcc-40c4-412f-8ea2-7013ddc5c955 None admin] PATH_INFO: `/v3/users/ca5d3774b93a41d1a8b95796b7899d4c` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:33.038796 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef205fcc-40c4-412f-8ea2-7013ddc5c955 None admin] RBAC: Authorizing `identity:get_user(user_id=ca5d3774b93a41d1a8b95796b7899d4c)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:33.040522 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef205fcc-40c4-412f-8ea2-7013ddc5c955 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:33.042602 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 21/36] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Aug 6 01:19:32 2022] GET /identity/v3/users/ca5d3774b93a41d1a8b95796b7899d4c => generated 247 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:19:33.044084 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e80605-5291-4985-884c-839f3477e26e None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:33.044578 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e80605-5291-4985-884c-839f3477e26e None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:33.050042 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e80605-5291-4985-884c-839f3477e26e None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:33.051975 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 22/37] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:19:33 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:19:33.052991 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e80605-5291-4985-884c-839f3477e26e None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:33.058490 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:33.069329 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9778124e-828e-4e74-b6ba-9ac2d1d6c63f None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:33.111178 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b93fea-a20b-4f41-abf8-f6491bd00fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:33.112041 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b93fea-a20b-4f41-abf8-f6491bd00fa7 None admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:33.112481 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b93fea-a20b-4f41-abf8-f6491bd00fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:33.112740 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b93fea-a20b-4f41-abf8-f6491bd00fa7 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:33.113680 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b93fea-a20b-4f41-abf8-f6491bd00fa7 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:33.115266 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b93fea-a20b-4f41-abf8-f6491bd00fa7 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:33.132735 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 16/38] 10.222.0.26 () {60 vars in 1131 bytes} [Sat Aug 6 01:19:32 2022] POST /identity/v3/roles => generated 225 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:19:33.139730 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f644df7-b729-475c-aa07-5fab5490dde8 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:33.139730 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f644df7-b729-475c-aa07-5fab5490dde8 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:33.140078 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f644df7-b729-475c-aa07-5fab5490dde8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:33.178239 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-0f644df7-b729-475c-aa07-5fab5490dde8 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:19:33.179299 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e80605-5291-4985-884c-839f3477e26e None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:33.179809 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-0f644df7-b729-475c-aa07-5fab5490dde8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:19:33.224053 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f9eb06-65e8-443d-a781-56fb28aa58ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:33.224829 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f9eb06-65e8-443d-a781-56fb28aa58ea None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:33.226399 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f9eb06-65e8-443d-a781-56fb28aa58ea None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:33.226399 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f9eb06-65e8-443d-a781-56fb28aa58ea None admin] PATH_INFO: `/v3/domains/default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:33.227037 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f9eb06-65e8-443d-a781-56fb28aa58ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:33.228523 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f9eb06-65e8-443d-a781-56fb28aa58ea None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:33.230504 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 23/39] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:19:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:19:33.237027 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f9eb06-65e8-443d-a781-56fb28aa58ea None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:33.242173 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:33.278088 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 17/40] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:19:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:19:33.280464 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffedaa2-0f6e-45cc-a44a-ae00313b88b2 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:33.280943 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffedaa2-0f6e-45cc-a44a-ae00313b88b2 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:33.281395 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffedaa2-0f6e-45cc-a44a-ae00313b88b2 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:33.283191 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 18/41] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:19:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:19:33.289992 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f573e962-cd8c-4bc4-9d25-59afdc93c388 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:33.290473 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f573e962-cd8c-4bc4-9d25-59afdc93c388 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:33.290981 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f573e962-cd8c-4bc4-9d25-59afdc93c388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:33.311720 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-f573e962-cd8c-4bc4-9d25-59afdc93c388 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:19:33.313384 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-f573e962-cd8c-4bc4-9d25-59afdc93c388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:19:33.372295 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:33.378261 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f9eb06-65e8-443d-a781-56fb28aa58ea None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:33.404341 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 19/42] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:19:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 01:19:33.404657 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f573e962-cd8c-4bc4-9d25-59afdc93c388 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:33.415901 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176e6e01-157b-4fe2-89e8-8178cbe32164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:33.416574 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176e6e01-157b-4fe2-89e8-8178cbe32164 None admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:33.416861 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176e6e01-157b-4fe2-89e8-8178cbe32164 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:33.417175 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176e6e01-157b-4fe2-89e8-8178cbe32164 None admin] PATH_INFO: `/v3/domains/default/users/ca5d3774b93a41d1a8b95796b7899d4c/roles/4559b9a0837249f895a13c70096f8d38` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:33.419208 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176e6e01-157b-4fe2-89e8-8178cbe32164 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=ca5d3774b93a41d1a8b95796b7899d4c, role_id=4559b9a0837249f895a13c70096f8d38)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:33.420957 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176e6e01-157b-4fe2-89e8-8178cbe32164 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:33.505423 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f573e962-cd8c-4bc4-9d25-59afdc93c388 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:33.537464 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 24/43] 10.222.0.26 () {58 vars in 1361 bytes} [Sat Aug 6 01:19:33 2022] PUT /identity/v3/domains/default/users/ca5d3774b93a41d1a8b95796b7899d4c/roles/4559b9a0837249f895a13c70096f8d38 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:19:33.538522 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c010ce0-ea61-4058-a53d-129185bbf23d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:33.538679 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176e6e01-157b-4fe2-89e8-8178cbe32164 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:33.539139 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c010ce0-ea61-4058-a53d-129185bbf23d None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:33.539428 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c010ce0-ea61-4058-a53d-129185bbf23d None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:33.539705 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c010ce0-ea61-4058-a53d-129185bbf23d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:33.540949 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c010ce0-ea61-4058-a53d-129185bbf23d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:33.542405 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c010ce0-ea61-4058-a53d-129185bbf23d None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:33.544106 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 20/44] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:19:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:19:33.550797 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c010ce0-ea61-4058-a53d-129185bbf23d None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:33.602597 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:33.622841 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:33.665556 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c010ce0-ea61-4058-a53d-129185bbf23d None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:33.670706 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176e6e01-157b-4fe2-89e8-8178cbe32164 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:33.709092 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599f2bb0-2876-4784-8ebc-2b3c95facbd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:33.709729 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599f2bb0-2876-4784-8ebc-2b3c95facbd9 None admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:33.710018 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599f2bb0-2876-4784-8ebc-2b3c95facbd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:33.710302 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599f2bb0-2876-4784-8ebc-2b3c95facbd9 None admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:33.710997 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599f2bb0-2876-4784-8ebc-2b3c95facbd9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:33.712462 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599f2bb0-2876-4784-8ebc-2b3c95facbd9 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:33.714789 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35f418f-18af-4942-bb2c-aa67f619b55f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:33.715593 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35f418f-18af-4942-bb2c-aa67f619b55f None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:33.716025 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35f418f-18af-4942-bb2c-aa67f619b55f None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:33.716452 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35f418f-18af-4942-bb2c-aa67f619b55f None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:33.717410 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35f418f-18af-4942-bb2c-aa67f619b55f None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:33.719278 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35f418f-18af-4942-bb2c-aa67f619b55f None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:33.740104 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 25/45] 10.222.0.26 () {60 vars in 1131 bytes} [Sat Aug 6 01:19:33 2022] POST /identity/v3/roles => generated 223 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:19:33.744710 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 21/46] 10.222.0.26 () {60 vars in 1141 bytes} [Sat Aug 6 01:19:33 2022] POST /identity/v3/projects => generated 312 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:19:35.540243 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672c457b-2a7e-46b7-978a-07f8428cfdda None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:35.540874 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672c457b-2a7e-46b7-978a-07f8428cfdda None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:35.540874 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672c457b-2a7e-46b7-978a-07f8428cfdda None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:35.542288 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 22/47] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:19:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:19:35.547819 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6187ea0a-0eb1-4b9b-9cf9-9d91c201df67 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:35.548230 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6187ea0a-0eb1-4b9b-9cf9-9d91c201df67 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:35.548385 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6187ea0a-0eb1-4b9b-9cf9-9d91c201df67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:35.566988 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-6187ea0a-0eb1-4b9b-9cf9-9d91c201df67 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:19:35.568495 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-6187ea0a-0eb1-4b9b-9cf9-9d91c201df67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:19:35.633575 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 26/48] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:19:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:19:35.656677 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672c457b-2a7e-46b7-978a-07f8428cfdda None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:35.752573 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:35.757867 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672c457b-2a7e-46b7-978a-07f8428cfdda None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:35.791161 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53de04bf-c572-45bd-859d-5a96ad937a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:35.791804 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53de04bf-c572-45bd-859d-5a96ad937a4c None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:35.792093 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53de04bf-c572-45bd-859d-5a96ad937a4c None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:35.792370 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53de04bf-c572-45bd-859d-5a96ad937a4c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:35.800553 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dabec7-a5d9-4799-b315-de6d575954aa None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:35.800853 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dabec7-a5d9-4799-b315-de6d575954aa None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:35.801170 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dabec7-a5d9-4799-b315-de6d575954aa None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:35.802685 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 27/49] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:19:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:19:35.803624 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53de04bf-c572-45bd-859d-5a96ad937a4c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:19:35.807371 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d47083c-262e-494a-b4c6-2cd594ed34e2 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:35.807662 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d47083c-262e-494a-b4c6-2cd594ed34e2 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:35.807947 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d47083c-262e-494a-b4c6-2cd594ed34e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:35.808605 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-53de04bf-c572-45bd-859d-5a96ad937a4c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:19:35.808605 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:19:35.808605 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:19:35.808605 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:19:35.808605 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:19:35.808605 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:19:35.808605 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:19:35.808605 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:19:35.808605 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:19:35.808605 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:19:35.808605 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:19:35.808605 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:19:35.808605 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:19:35.808605 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:19:35.808605 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:19:35.808605 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:19:35.809629 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:19:35.809629 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:19:35.809629 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:19:35.809629 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:19:35.809629 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:19:35.809629 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:19:35.809629 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:19:35.809629 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:19:35.809629 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:19:35.809629 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:19:35.809629 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:19:35.809629 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:19:35.809629 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:19:35.809629 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:19:35.809629 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:19:35.809629 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:19:35.810413 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:19:35.810413 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:19:35.810413 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:19:35.810413 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:19:35.810413 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:19:35.810413 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:19:35.810413 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:19:35.810413 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:19:35.810413 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:19:35.810413 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:19:35.810413 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:19:35.810413 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:19:35.810413 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 23/50] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Aug 6 01:19:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:19:35.815520 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53de04bf-c572-45bd-859d-5a96ad937a4c None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:35.826979 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-7d47083c-262e-494a-b4c6-2cd594ed34e2 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:19:35.828191 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-7d47083c-262e-494a-b4c6-2cd594ed34e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:19:35.878279 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:35.898912 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 28/51] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:19:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 01:19:35.918986 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53de04bf-c572-45bd-859d-5a96ad937a4c None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:35.921372 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d47083c-262e-494a-b4c6-2cd594ed34e2 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:35.952562 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820f2dd4-e2a0-42df-834d-5637d341ddbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:35.953286 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820f2dd4-e2a0-42df-834d-5637d341ddbc None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:35.953580 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820f2dd4-e2a0-42df-834d-5637d341ddbc None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:35.953864 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820f2dd4-e2a0-42df-834d-5637d341ddbc None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:35.954588 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820f2dd4-e2a0-42df-834d-5637d341ddbc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:19:35.954898 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820f2dd4-e2a0-42df-834d-5637d341ddbc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:35.956291 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820f2dd4-e2a0-42df-834d-5637d341ddbc None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:35.968598 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 24/52] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Aug 6 01:19:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:19:35.974271 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820f2dd4-e2a0-42df-834d-5637d341ddbc None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:35.995405 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:36.020102 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d47083c-262e-494a-b4c6-2cd594ed34e2 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:36.054015 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a564747-ed27-483a-86ce-0ed5fd070012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:36.054665 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a564747-ed27-483a-86ce-0ed5fd070012 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:36.054956 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a564747-ed27-483a-86ce-0ed5fd070012 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:36.055244 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a564747-ed27-483a-86ce-0ed5fd070012 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:36.056706 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a564747-ed27-483a-86ce-0ed5fd070012 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:36.058430 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a564747-ed27-483a-86ce-0ed5fd070012 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:36.060335 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 29/53] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:19:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:19:36.067065 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a564747-ed27-483a-86ce-0ed5fd070012 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:36.074725 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820f2dd4-e2a0-42df-834d-5637d341ddbc None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:36.109019 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5edd45-bc7d-4e92-840f-42c62e03ef3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:36.109700 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5edd45-bc7d-4e92-840f-42c62e03ef3d None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:36.109991 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5edd45-bc7d-4e92-840f-42c62e03ef3d None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:36.110270 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5edd45-bc7d-4e92-840f-42c62e03ef3d None admin] PATH_INFO: `/v3/users/ca5d3774b93a41d1a8b95796b7899d4c` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:36.111447 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5edd45-bc7d-4e92-840f-42c62e03ef3d None admin] RBAC: Authorizing `identity:get_user(user_id=ca5d3774b93a41d1a8b95796b7899d4c)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:36.112880 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5edd45-bc7d-4e92-840f-42c62e03ef3d None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:36.114652 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 25/54] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Aug 6 01:19:35 2022] GET /identity/v3/users/ca5d3774b93a41d1a8b95796b7899d4c => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:19:36.119775 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5edd45-bc7d-4e92-840f-42c62e03ef3d None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:36.120829 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:36.168202 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a564747-ed27-483a-86ce-0ed5fd070012 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:36.203403 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39ecf54-e72e-487e-979d-6c87c521d561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:36.204039 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39ecf54-e72e-487e-979d-6c87c521d561 None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:36.204329 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39ecf54-e72e-487e-979d-6c87c521d561 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:36.204610 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39ecf54-e72e-487e-979d-6c87c521d561 None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:36.205331 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39ecf54-e72e-487e-979d-6c87c521d561 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:36.206783 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39ecf54-e72e-487e-979d-6c87c521d561 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:36.207861 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:36.222107 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5edd45-bc7d-4e92-840f-42c62e03ef3d None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:36.240793 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 30/55] 10.222.0.26 () {60 vars in 1140 bytes} [Sat Aug 6 01:19:36 2022] POST /identity/v3/projects => generated 298 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:19:36.257918 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326419ed-4406-4b5d-a499-1e6abe7d53d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:36.258602 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326419ed-4406-4b5d-a499-1e6abe7d53d3 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:36.258892 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326419ed-4406-4b5d-a499-1e6abe7d53d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:36.259174 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326419ed-4406-4b5d-a499-1e6abe7d53d3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:36.260428 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326419ed-4406-4b5d-a499-1e6abe7d53d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:36.261960 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326419ed-4406-4b5d-a499-1e6abe7d53d3 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:36.269027 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 26/56] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:19:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:19:36.270336 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39ecf54-e72e-487e-979d-6c87c521d561 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:36.379538 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39ecf54-e72e-487e-979d-6c87c521d561 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:36.381962 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:36.419006 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3ea0f2-b949-431d-8196-f75a140d602e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:36.419643 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3ea0f2-b949-431d-8196-f75a140d602e None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:36.419980 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3ea0f2-b949-431d-8196-f75a140d602e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:36.420259 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3ea0f2-b949-431d-8196-f75a140d602e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:36.421246 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3ea0f2-b949-431d-8196-f75a140d602e None admin] RBAC: Adding query filter params (role.id=4559b9a0837249f895a13c70096f8d38, scope.domain.id=default, user.id=ca5d3774b93a41d1a8b95796b7899d4c) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:19:36.421557 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3ea0f2-b949-431d-8196-f75a140d602e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:36.423002 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3ea0f2-b949-431d-8196-f75a140d602e None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:36.428958 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 31/57] 10.222.0.26 () {56 vars in 1322 bytes} [Sat Aug 6 01:19:36 2022] GET /identity/v3/role_assignments?user.id=ca5d3774b93a41d1a8b95796b7899d4c&scope.domain.id=default&role.id=4559b9a0837249f895a13c70096f8d38 => generated 530 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:19:38.243517 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca66d37b-eacb-48ba-bccb-6cf11df876c8 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:38.244027 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca66d37b-eacb-48ba-bccb-6cf11df876c8 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:38.244114 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca66d37b-eacb-48ba-bccb-6cf11df876c8 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:38.245459 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 27/58] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:19:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:19:38.251024 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0daf452-4478-4f33-ba3a-a1e14398cc62 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:38.251442 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0daf452-4478-4f33-ba3a-a1e14398cc62 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:38.251596 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0daf452-4478-4f33-ba3a-a1e14398cc62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:38.269255 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-e0daf452-4478-4f33-ba3a-a1e14398cc62 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:19:38.270452 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-e0daf452-4478-4f33-ba3a-a1e14398cc62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:19:38.334518 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 32/59] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:19:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:19:38.358633 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca66d37b-eacb-48ba-bccb-6cf11df876c8 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:38.458448 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca66d37b-eacb-48ba-bccb-6cf11df876c8 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:38.491146 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b133a6bc-98cd-41c6-91f2-55e2f50d756c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:38.491785 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b133a6bc-98cd-41c6-91f2-55e2f50d756c None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:38.492073 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b133a6bc-98cd-41c6-91f2-55e2f50d756c None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:38.492351 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b133a6bc-98cd-41c6-91f2-55e2f50d756c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:38.493587 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b133a6bc-98cd-41c6-91f2-55e2f50d756c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:38.494998 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b133a6bc-98cd-41c6-91f2-55e2f50d756c None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:38.496582 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 28/60] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:19:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:19:38.502990 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0daf452-4478-4f33-ba3a-a1e14398cc62 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:38.596566 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:38.600286 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0daf452-4478-4f33-ba3a-a1e14398cc62 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:38.633750 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a424b5a4-b9a3-4566-8427-1fb1c8afed2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:38.634364 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a424b5a4-b9a3-4566-8427-1fb1c8afed2e None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:38.634653 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a424b5a4-b9a3-4566-8427-1fb1c8afed2e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:38.634939 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a424b5a4-b9a3-4566-8427-1fb1c8afed2e None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:38.635632 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a424b5a4-b9a3-4566-8427-1fb1c8afed2e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:38.637095 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a424b5a4-b9a3-4566-8427-1fb1c8afed2e None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:38.663792 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 33/61] 10.222.0.26 () {60 vars in 1132 bytes} [Sat Aug 6 01:19:38 2022] POST /identity/v3/users => generated 275 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:19:41.785967 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd66ede3-259b-45f2-9887-e4b3e6954a36 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:41.786469 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd66ede3-259b-45f2-9887-e4b3e6954a36 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:41.786567 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd66ede3-259b-45f2-9887-e4b3e6954a36 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:41.788030 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 29/62] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:19:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:19:41.811526 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dfdbed-15f5-43c3-899b-fecbf15491de None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:41.812022 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dfdbed-15f5-43c3-899b-fecbf15491de None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:41.812094 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dfdbed-15f5-43c3-899b-fecbf15491de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:41.832466 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-91dfdbed-15f5-43c3-899b-fecbf15491de None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:19:41.834075 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-91dfdbed-15f5-43c3-899b-fecbf15491de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:19:41.923329 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 34/63] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:19:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 01:19:41.952643 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd66ede3-259b-45f2-9887-e4b3e6954a36 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:41.992215 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:42.023517 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8e2c18-8eee-427b-864b-cce64fe108fb None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:42.023898 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8e2c18-8eee-427b-864b-cce64fe108fb None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:42.024424 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8e2c18-8eee-427b-864b-cce64fe108fb None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:42.026024 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 35/64] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:19:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 01:19:42.038872 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cf7e01-4d10-4cff-9aae-fc53d8c49e9a None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:42.039439 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cf7e01-4d10-4cff-9aae-fc53d8c49e9a None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:42.039919 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cf7e01-4d10-4cff-9aae-fc53d8c49e9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:42.062866 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-70cf7e01-4d10-4cff-9aae-fc53d8c49e9a None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:19:42.064473 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-70cf7e01-4d10-4cff-9aae-fc53d8c49e9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:19:42.097757 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd66ede3-259b-45f2-9887-e4b3e6954a36 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:42.134779 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5d04a2-fe19-44c7-921d-5e06a93efe2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:42.134975 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 36/65] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:19:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 01:19:42.135877 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5d04a2-fe19-44c7-921d-5e06a93efe2a None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:42.136260 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0dae66-e29d-44f1-9246-b3b5ada29e2b None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:42.136465 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5d04a2-fe19-44c7-921d-5e06a93efe2a None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:42.136845 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5d04a2-fe19-44c7-921d-5e06a93efe2a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:42.137021 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0dae66-e29d-44f1-9246-b3b5ada29e2b None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:42.137478 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0dae66-e29d-44f1-9246-b3b5ada29e2b None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:42.139711 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 37/66] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:19:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:19:42.144669 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049e1425-91f8-4d64-a4b0-2e353f2f482b None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:42.145245 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049e1425-91f8-4d64-a4b0-2e353f2f482b None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:42.146485 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049e1425-91f8-4d64-a4b0-2e353f2f482b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:42.166186 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5d04a2-fe19-44c7-921d-5e06a93efe2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:19:42.172085 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-8e5d04a2-fe19-44c7-921d-5e06a93efe2a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 06 01:19:42.172085 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:19:42.172085 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:19:42.172085 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:19:42.172085 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:19:42.172085 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:19:42.172085 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:19:42.172085 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:19:42.172085 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:19:42.172085 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:19:42.172085 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:19:42.172085 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:19:42.172085 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:19:42.172085 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:19:42.172085 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:19:42.172085 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:19:42.172926 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:19:42.172926 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:19:42.172926 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:19:42.172926 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:19:42.172926 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:19:42.172926 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:19:42.172926 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:19:42.172926 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:19:42.172926 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:19:42.172926 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:19:42.172926 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:19:42.172926 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:19:42.172926 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:19:42.172926 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:19:42.172926 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:19:42.172926 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:19:42.173715 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:19:42.173715 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:19:42.173715 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:19:42.173715 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:19:42.173715 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:19:42.173715 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:19:42.173715 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:19:42.173715 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:19:42.173715 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:19:42.173715 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:19:42.173715 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 06 01:19:42.173715 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:19:42.175544 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-049e1425-91f8-4d64-a4b0-2e353f2f482b None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:19:42.176965 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-049e1425-91f8-4d64-a4b0-2e353f2f482b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:19:42.178205 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 30/67] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Aug 6 01:19:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:19:42.179063 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5d04a2-fe19-44c7-921d-5e06a93efe2a None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:42.188748 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:42.264230 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 38/68] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:19:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:19:42.265484 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049e1425-91f8-4d64-a4b0-2e353f2f482b None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:42.282235 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:42.295405 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5d04a2-fe19-44c7-921d-5e06a93efe2a None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:42.331178 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cd838b-e01d-4f0c-93f6-3878f3a4afb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:42.331828 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cd838b-e01d-4f0c-93f6-3878f3a4afb8 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:42.332116 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cd838b-e01d-4f0c-93f6-3878f3a4afb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:42.332392 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cd838b-e01d-4f0c-93f6-3878f3a4afb8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:42.343246 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cd838b-e01d-4f0c-93f6-3878f3a4afb8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:19:42.348171 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-12cd838b-e01d-4f0c-93f6-3878f3a4afb8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:19:42.348171 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:19:42.348171 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:19:42.348171 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:19:42.348171 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:19:42.348171 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:19:42.348171 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:19:42.348171 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:19:42.348171 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:19:42.348171 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:19:42.348171 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:19:42.348171 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:19:42.348171 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:19:42.348171 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:19:42.348171 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:19:42.348171 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:19:42.349263 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:19:42.349263 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:19:42.349263 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:19:42.349263 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:19:42.349263 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:19:42.349263 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:19:42.349263 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:19:42.349263 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:19:42.349263 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:19:42.349263 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:19:42.349263 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:19:42.349263 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:19:42.349263 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:19:42.349263 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:19:42.349263 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:19:42.349263 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:19:42.350225 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:19:42.350225 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:19:42.350225 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:19:42.350225 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:19:42.350225 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:19:42.350225 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:19:42.350225 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:19:42.350225 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:19:42.350225 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:19:42.350225 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:19:42.350225 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:19:42.350225 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:19:42.350225 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 31/69] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Aug 6 01:19:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:19:42.351266 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5121c52d-c3dd-47da-870f-d69313a276e5 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:42.351266 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5121c52d-c3dd-47da-870f-d69313a276e5 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:42.351403 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5121c52d-c3dd-47da-870f-d69313a276e5 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:42.352695 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 32/70] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:19:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:19:42.353291 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5121c52d-c3dd-47da-870f-d69313a276e5 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:42.391316 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049e1425-91f8-4d64-a4b0-2e353f2f482b None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:42.425782 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee12ce8f-45ae-4a5a-a3f1-1ee8ddb4c3ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:42.426416 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee12ce8f-45ae-4a5a-a3f1-1ee8ddb4c3ef None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:42.426705 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee12ce8f-45ae-4a5a-a3f1-1ee8ddb4c3ef None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:42.426994 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee12ce8f-45ae-4a5a-a3f1-1ee8ddb4c3ef None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:42.427725 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee12ce8f-45ae-4a5a-a3f1-1ee8ddb4c3ef None admin] RBAC: Adding query filter params (name=member) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:19:42.428042 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee12ce8f-45ae-4a5a-a3f1-1ee8ddb4c3ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:42.429472 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee12ce8f-45ae-4a5a-a3f1-1ee8ddb4c3ef None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:42.438238 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 39/71] 10.222.0.26 () {56 vars in 1101 bytes} [Sat Aug 6 01:19:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:19:42.439427 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0d5ae6-c4e3-4bb4-996a-bd88a669db8b None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:42.439720 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0d5ae6-c4e3-4bb4-996a-bd88a669db8b None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:42.440004 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0d5ae6-c4e3-4bb4-996a-bd88a669db8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:42.452742 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:42.458233 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-8b0d5ae6-c4e3-4bb4-996a-bd88a669db8b None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:19:42.459452 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-8b0d5ae6-c4e3-4bb4-996a-bd88a669db8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:19:42.464066 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5121c52d-c3dd-47da-870f-d69313a276e5 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:42.498735 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064fe3fb-5f1f-4dd5-ac55-2a5e3ed0b1ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:42.499375 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064fe3fb-5f1f-4dd5-ac55-2a5e3ed0b1ca None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:42.499663 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064fe3fb-5f1f-4dd5-ac55-2a5e3ed0b1ca None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:42.499944 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064fe3fb-5f1f-4dd5-ac55-2a5e3ed0b1ca None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:42.510195 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064fe3fb-5f1f-4dd5-ac55-2a5e3ed0b1ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:19:42.515805 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-064fe3fb-5f1f-4dd5-ac55-2a5e3ed0b1ca None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 01:19:42.515805 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:19:42.515805 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:19:42.515805 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:19:42.515805 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:19:42.515805 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:19:42.515805 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:19:42.515805 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:19:42.515805 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:19:42.515805 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:19:42.515805 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:19:42.515805 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:19:42.515805 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:19:42.515805 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:19:42.515805 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:19:42.515805 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:19:42.516735 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:19:42.516735 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:19:42.516735 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:19:42.516735 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:19:42.516735 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:19:42.516735 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:19:42.516735 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:19:42.516735 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:19:42.516735 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:19:42.516735 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:19:42.516735 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:19:42.516735 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:19:42.516735 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:19:42.516735 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:19:42.516735 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:19:42.516735 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:19:42.517522 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:19:42.517522 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:19:42.517522 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:19:42.517522 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:19:42.517522 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:19:42.517522 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:19:42.517522 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:19:42.517522 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:19:42.517522 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:19:42.517522 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:19:42.517522 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 01:19:42.517522 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:19:42.517522 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 33/72] 10.222.0.26 () {56 vars in 1114 bytes} [Sat Aug 6 01:19:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:19:42.518270 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af185325-4f22-44fe-8085-e843145bfc18 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:42.518898 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af185325-4f22-44fe-8085-e843145bfc18 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:42.518898 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af185325-4f22-44fe-8085-e843145bfc18 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:42.520204 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 34/73] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:19:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:19:42.522286 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af185325-4f22-44fe-8085-e843145bfc18 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:42.526657 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:42.531485 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 40/74] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:19:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:19:42.538898 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0d5ae6-c4e3-4bb4-996a-bd88a669db8b None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:42.628930 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af185325-4f22-44fe-8085-e843145bfc18 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:42.636721 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0d5ae6-c4e3-4bb4-996a-bd88a669db8b None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:42.661779 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7077f6-b23a-49f4-9ab4-d59df1bf340e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:42.662397 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7077f6-b23a-49f4-9ab4-d59df1bf340e None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:42.662684 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7077f6-b23a-49f4-9ab4-d59df1bf340e None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:42.662962 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7077f6-b23a-49f4-9ab4-d59df1bf340e None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:42.663679 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7077f6-b23a-49f4-9ab4-d59df1bf340e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:19:42.663985 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7077f6-b23a-49f4-9ab4-d59df1bf340e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:42.665412 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7077f6-b23a-49f4-9ab4-d59df1bf340e None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:42.673998 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 35/75] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Aug 6 01:19:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:19:42.674872 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7077f6-b23a-49f4-9ab4-d59df1bf340e None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:42.679958 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd97662-791d-4817-beec-cb051a1fd167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:42.680604 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd97662-791d-4817-beec-cb051a1fd167 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:42.680888 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd97662-791d-4817-beec-cb051a1fd167 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:42.681195 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd97662-791d-4817-beec-cb051a1fd167 None admin] PATH_INFO: `/v3/users/1cf42c5c30ba4b9a93fc4cdb9713c6cc` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:42.684180 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:42.698842 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd97662-791d-4817-beec-cb051a1fd167 None admin] RBAC: Authorizing `identity:get_user(user_id=1cf42c5c30ba4b9a93fc4cdb9713c6cc)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:42.700409 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd97662-791d-4817-beec-cb051a1fd167 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:42.702290 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 41/76] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Aug 6 01:19:42 2022] GET /identity/v3/users/1cf42c5c30ba4b9a93fc4cdb9713c6cc => generated 275 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:19:42.703490 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2571a12b-86ab-4271-83fc-1cb135a245ae None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:42.703783 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2571a12b-86ab-4271-83fc-1cb135a245ae None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:42.704063 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2571a12b-86ab-4271-83fc-1cb135a245ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:42.720840 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-2571a12b-86ab-4271-83fc-1cb135a245ae None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:19:42.722280 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-2571a12b-86ab-4271-83fc-1cb135a245ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:19:42.747036 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:42.788410 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7077f6-b23a-49f4-9ab4-d59df1bf340e None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:42.792037 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 42/77] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:19:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:19:42.792879 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2571a12b-86ab-4271-83fc-1cb135a245ae None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:42.823059 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a459d9-f7b2-4de8-879b-e9966dfb6869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:42.823678 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a459d9-f7b2-4de8-879b-e9966dfb6869 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:42.823957 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a459d9-f7b2-4de8-879b-e9966dfb6869 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:42.824234 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a459d9-f7b2-4de8-879b-e9966dfb6869 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:42.824945 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a459d9-f7b2-4de8-879b-e9966dfb6869 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:19:42.825274 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a459d9-f7b2-4de8-879b-e9966dfb6869 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:42.827501 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a459d9-f7b2-4de8-879b-e9966dfb6869 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:42.835268 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 36/78] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Aug 6 01:19:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:19:42.836245 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a459d9-f7b2-4de8-879b-e9966dfb6869 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:42.893413 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2571a12b-86ab-4271-83fc-1cb135a245ae None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:42.916994 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:42.929129 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357bb8cb-ea91-4011-b309-5ace4e36d338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:42.929781 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357bb8cb-ea91-4011-b309-5ace4e36d338 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:42.930074 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357bb8cb-ea91-4011-b309-5ace4e36d338 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:42.930361 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357bb8cb-ea91-4011-b309-5ace4e36d338 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:42.933949 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a459d9-f7b2-4de8-879b-e9966dfb6869 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:42.940517 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357bb8cb-ea91-4011-b309-5ace4e36d338 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:19:42.945981 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-357bb8cb-ea91-4011-b309-5ace4e36d338 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 06 01:19:42.945981 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:19:42.945981 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:19:42.945981 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:19:42.945981 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:19:42.945981 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:19:42.945981 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:19:42.945981 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:19:42.945981 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:19:42.945981 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:19:42.945981 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:19:42.945981 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:19:42.945981 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:19:42.945981 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:19:42.945981 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:19:42.945981 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:19:42.946784 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:19:42.946784 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:19:42.946784 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:19:42.946784 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:19:42.946784 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:19:42.946784 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:19:42.946784 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:19:42.946784 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:19:42.946784 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:19:42.946784 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:19:42.946784 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:19:42.946784 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:19:42.946784 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:19:42.946784 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:19:42.946784 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:19:42.946784 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:19:42.947735 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:19:42.947735 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:19:42.947735 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:19:42.947735 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:19:42.947735 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:19:42.947735 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:19:42.947735 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:19:42.947735 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:19:42.947735 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:19:42.947735 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:19:42.947735 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 06 01:19:42.947735 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:19:42.947735 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 43/79] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Aug 6 01:19:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:19:42.948538 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357bb8cb-ea91-4011-b309-5ace4e36d338 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:42.963883 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:42.972593 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7688e714-e619-4d77-96a7-ddc10b02b45e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:42.973272 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7688e714-e619-4d77-96a7-ddc10b02b45e None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:42.973574 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7688e714-e619-4d77-96a7-ddc10b02b45e None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:42.973836 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7688e714-e619-4d77-96a7-ddc10b02b45e None admin] PATH_INFO: `/v3/users/ca5d3774b93a41d1a8b95796b7899d4c` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:42.975015 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7688e714-e619-4d77-96a7-ddc10b02b45e None admin] RBAC: Authorizing `identity:get_user(user_id=ca5d3774b93a41d1a8b95796b7899d4c)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:42.976717 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7688e714-e619-4d77-96a7-ddc10b02b45e None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:42.978343 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 37/80] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Aug 6 01:19:42 2022] GET /identity/v3/users/ca5d3774b93a41d1a8b95796b7899d4c => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:19:42.979096 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7688e714-e619-4d77-96a7-ddc10b02b45e None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:43.052427 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357bb8cb-ea91-4011-b309-5ace4e36d338 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:43.079653 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7688e714-e619-4d77-96a7-ddc10b02b45e None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:43.086416 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b23b34-543a-4a4b-ac50-41377063c842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:43.087038 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b23b34-543a-4a4b-ac50-41377063c842 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:43.087317 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b23b34-543a-4a4b-ac50-41377063c842 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:43.087593 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b23b34-543a-4a4b-ac50-41377063c842 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:43.114112 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e0ccf3-7123-433f-a171-286d66dafebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:43.114758 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e0ccf3-7123-433f-a171-286d66dafebd None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:43.115052 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e0ccf3-7123-433f-a171-286d66dafebd None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:43.115333 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e0ccf3-7123-433f-a171-286d66dafebd None admin] PATH_INFO: `/v3/domains/default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:43.116506 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e0ccf3-7123-433f-a171-286d66dafebd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:43.118005 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e0ccf3-7123-433f-a171-286d66dafebd None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:43.119596 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 38/81] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:19:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:19:43.124878 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e0ccf3-7123-433f-a171-286d66dafebd None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:43.144605 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:43.160521 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b23b34-543a-4a4b-ac50-41377063c842 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:43.162039 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b23b34-543a-4a4b-ac50-41377063c842 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:43.183416 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:43.221924 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e0ccf3-7123-433f-a171-286d66dafebd None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:43.255023 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e429989-126b-4734-8649-99b75c8f5cc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:43.255779 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e429989-126b-4734-8649-99b75c8f5cc1 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:43.256093 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e429989-126b-4734-8649-99b75c8f5cc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:43.256358 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e429989-126b-4734-8649-99b75c8f5cc1 None admin] PATH_INFO: `/v3/users/1cf42c5c30ba4b9a93fc4cdb9713c6cc` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:43.257577 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e429989-126b-4734-8649-99b75c8f5cc1 None admin] RBAC: Authorizing `identity:get_user(user_id=1cf42c5c30ba4b9a93fc4cdb9713c6cc)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:43.259042 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e429989-126b-4734-8649-99b75c8f5cc1 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:43.259503 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 44/82] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Aug 6 01:19:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 01:19:43.260333 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b23b34-543a-4a4b-ac50-41377063c842 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:43.260827 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 39/83] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Aug 6 01:19:43 2022] GET /identity/v3/users/1cf42c5c30ba4b9a93fc4cdb9713c6cc => generated 275 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:19:43.265914 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e429989-126b-4734-8649-99b75c8f5cc1 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:43.342606 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:43.359533 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b23b34-543a-4a4b-ac50-41377063c842 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:43.364998 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e429989-126b-4734-8649-99b75c8f5cc1 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:43.393301 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8cde9f-95a2-4e39-927e-6c5cbf7432a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:43.393929 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8cde9f-95a2-4e39-927e-6c5cbf7432a6 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:43.394213 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8cde9f-95a2-4e39-927e-6c5cbf7432a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:43.394497 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8cde9f-95a2-4e39-927e-6c5cbf7432a6 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:43.395099 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:43.395281 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8cde9f-95a2-4e39-927e-6c5cbf7432a6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:19:43.399588 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8cde9f-95a2-4e39-927e-6c5cbf7432a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:43.400388 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb0df3d-025e-40b7-a3bb-337072e113b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:43.401204 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb0df3d-025e-40b7-a3bb-337072e113b3 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:43.401313 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8cde9f-95a2-4e39-927e-6c5cbf7432a6 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:43.404348 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb0df3d-025e-40b7-a3bb-337072e113b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:43.404348 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb0df3d-025e-40b7-a3bb-337072e113b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:43.410807 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 45/84] 10.222.0.26 () {56 vars in 1101 bytes} [Sat Aug 6 01:19:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:19:43.411617 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8cde9f-95a2-4e39-927e-6c5cbf7432a6 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:43.471477 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb0df3d-025e-40b7-a3bb-337072e113b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:43.472947 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb0df3d-025e-40b7-a3bb-337072e113b3 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:43.511523 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8cde9f-95a2-4e39-927e-6c5cbf7432a6 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:43.543772 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c897a9e0-bc9d-4b47-b238-47431e65e388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:43.544396 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c897a9e0-bc9d-4b47-b238-47431e65e388 None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:43.545241 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c897a9e0-bc9d-4b47-b238-47431e65e388 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:43.545519 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c897a9e0-bc9d-4b47-b238-47431e65e388 None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:43.546206 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c897a9e0-bc9d-4b47-b238-47431e65e388 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:43.547638 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c897a9e0-bc9d-4b47-b238-47431e65e388 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:43.568778 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 46/85] 10.222.0.26 () {60 vars in 1132 bytes} [Sat Aug 6 01:19:43 2022] POST /identity/v3/users => generated 289 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:19:43.569612 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c897a9e0-bc9d-4b47-b238-47431e65e388 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:43.572074 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:43.593706 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 40/86] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Aug 6 01:19:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 01:19:43.593706 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb0df3d-025e-40b7-a3bb-337072e113b3 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:43.622480 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:43.698698 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c897a9e0-bc9d-4b47-b238-47431e65e388 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:43.705468 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb0df3d-025e-40b7-a3bb-337072e113b3 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:43.738830 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2911ba49-8fae-4efe-badd-2c7a8b2d9971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:43.739484 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2911ba49-8fae-4efe-badd-2c7a8b2d9971 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:43.739770 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2911ba49-8fae-4efe-badd-2c7a8b2d9971 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:43.740054 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2911ba49-8fae-4efe-badd-2c7a8b2d9971 None admin] PATH_INFO: `/v3/projects/730685c4f62b43c493d42b5897c99a02` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:43.741260 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2911ba49-8fae-4efe-badd-2c7a8b2d9971 None admin] RBAC: Authorizing `identity:get_project(project_id=730685c4f62b43c493d42b5897c99a02)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:43.742709 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2911ba49-8fae-4efe-badd-2c7a8b2d9971 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:43.744375 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 47/87] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Aug 6 01:19:43 2022] GET /identity/v3/projects/730685c4f62b43c493d42b5897c99a02 => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:19:43.745650 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28352543-8a1d-4ade-b1a7-9513a4a26cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:43.745957 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2911ba49-8fae-4efe-badd-2c7a8b2d9971 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:43.746251 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28352543-8a1d-4ade-b1a7-9513a4a26cec None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:43.746546 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28352543-8a1d-4ade-b1a7-9513a4a26cec None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:43.746830 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28352543-8a1d-4ade-b1a7-9513a4a26cec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:43.826669 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:43.829942 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28352543-8a1d-4ade-b1a7-9513a4a26cec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:43.831483 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28352543-8a1d-4ade-b1a7-9513a4a26cec None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:43.867402 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2911ba49-8fae-4efe-badd-2c7a8b2d9971 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:43.873339 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:43.907368 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6940f5d7-f7ef-498e-9bf9-adc0049e86d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:43.908048 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6940f5d7-f7ef-498e-9bf9-adc0049e86d5 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:43.908337 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6940f5d7-f7ef-498e-9bf9-adc0049e86d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:43.908610 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6940f5d7-f7ef-498e-9bf9-adc0049e86d5 None admin] PATH_INFO: `/v3/users/1cf42c5c30ba4b9a93fc4cdb9713c6cc` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:43.911032 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6940f5d7-f7ef-498e-9bf9-adc0049e86d5 None admin] RBAC: Authorizing `identity:get_user(user_id=1cf42c5c30ba4b9a93fc4cdb9713c6cc)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:43.912572 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6940f5d7-f7ef-498e-9bf9-adc0049e86d5 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:43.914671 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 48/88] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Aug 6 01:19:43 2022] GET /identity/v3/users/1cf42c5c30ba4b9a93fc4cdb9713c6cc => generated 275 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:19:43.915035 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6940f5d7-f7ef-498e-9bf9-adc0049e86d5 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:43.965253 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 41/89] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Aug 6 01:19:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 01:19:43.966037 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28352543-8a1d-4ade-b1a7-9513a4a26cec None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:44.059224 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6940f5d7-f7ef-498e-9bf9-adc0049e86d5 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:44.101872 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28352543-8a1d-4ade-b1a7-9513a4a26cec None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:44.107896 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fc3b86-044b-49bf-ae8e-7d73d69a2479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:44.108620 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fc3b86-044b-49bf-ae8e-7d73d69a2479 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:44.108914 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fc3b86-044b-49bf-ae8e-7d73d69a2479 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:44.109320 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fc3b86-044b-49bf-ae8e-7d73d69a2479 None admin] PATH_INFO: `/v3/projects/730685c4f62b43c493d42b5897c99a02` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:44.111133 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fc3b86-044b-49bf-ae8e-7d73d69a2479 None admin] RBAC: Authorizing `identity:get_project(project_id=730685c4f62b43c493d42b5897c99a02)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:44.113147 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fc3b86-044b-49bf-ae8e-7d73d69a2479 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:44.116055 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 49/90] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Aug 6 01:19:43 2022] GET /identity/v3/projects/730685c4f62b43c493d42b5897c99a02 => generated 298 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:19:44.117197 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fc3b86-044b-49bf-ae8e-7d73d69a2479 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:44.131710 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:44.135203 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:44.142741 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0125534e-8840-4d7b-83d4-2e0c55d28a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:44.143540 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0125534e-8840-4d7b-83d4-2e0c55d28a79 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:44.143919 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0125534e-8840-4d7b-83d4-2e0c55d28a79 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:44.144262 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0125534e-8840-4d7b-83d4-2e0c55d28a79 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:44.145161 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0125534e-8840-4d7b-83d4-2e0c55d28a79 None admin] RBAC: Adding query filter params (role.id=61b3b93d37a047e4bb8369b46fe20db9, scope.project.id=730685c4f62b43c493d42b5897c99a02, user.id=1cf42c5c30ba4b9a93fc4cdb9713c6cc) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:19:44.145542 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0125534e-8840-4d7b-83d4-2e0c55d28a79 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:44.147301 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0125534e-8840-4d7b-83d4-2e0c55d28a79 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:44.155867 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 42/91] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Aug 6 01:19:43 2022] GET /identity/v3/role_assignments?user.id=1cf42c5c30ba4b9a93fc4cdb9713c6cc&scope.project.id=730685c4f62b43c493d42b5897c99a02&role.id=61b3b93d37a047e4bb8369b46fe20db9 => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:19:44.156564 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0125534e-8840-4d7b-83d4-2e0c55d28a79 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:44.320725 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fc3b86-044b-49bf-ae8e-7d73d69a2479 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:44.369072 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0125534e-8840-4d7b-83d4-2e0c55d28a79 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:44.417023 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460d0a72-6bf3-4496-88cf-eede6f180fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:44.417720 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460d0a72-6bf3-4496-88cf-eede6f180fa3 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:44.418009 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460d0a72-6bf3-4496-88cf-eede6f180fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:44.418290 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460d0a72-6bf3-4496-88cf-eede6f180fa3 None admin] PATH_INFO: `/v3/projects/730685c4f62b43c493d42b5897c99a02` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:44.420716 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460d0a72-6bf3-4496-88cf-eede6f180fa3 None admin] RBAC: Authorizing `identity:get_project(project_id=730685c4f62b43c493d42b5897c99a02)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:44.422874 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe84eb8-0f7b-40b3-8a6d-1bc7c97d4d45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:44.423517 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe84eb8-0f7b-40b3-8a6d-1bc7c97d4d45 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:44.423801 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe84eb8-0f7b-40b3-8a6d-1bc7c97d4d45 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:44.424080 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe84eb8-0f7b-40b3-8a6d-1bc7c97d4d45 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:44.425712 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460d0a72-6bf3-4496-88cf-eede6f180fa3 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:44.427694 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 43/92] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Aug 6 01:19:44 2022] GET /identity/v3/projects/730685c4f62b43c493d42b5897c99a02 => generated 298 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:19:44.428309 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460d0a72-6bf3-4496-88cf-eede6f180fa3 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:44.536637 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe84eb8-0f7b-40b3-8a6d-1bc7c97d4d45 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:44.538350 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe84eb8-0f7b-40b3-8a6d-1bc7c97d4d45 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:44.547491 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:44.590233 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:44.652940 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 50/93] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Aug 6 01:19:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 536 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 01:19:44.653809 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe84eb8-0f7b-40b3-8a6d-1bc7c97d4d45 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:44.695912 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460d0a72-6bf3-4496-88cf-eede6f180fa3 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:44.765306 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe84eb8-0f7b-40b3-8a6d-1bc7c97d4d45 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:44.773517 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:44.805496 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5f79ec-6cc6-463c-a9f2-b50b0b15a6ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:44.806490 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332c4164-5734-4b47-9b0b-f3e616e484cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:44.807342 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5f79ec-6cc6-463c-a9f2-b50b0b15a6ad None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:44.808225 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332c4164-5734-4b47-9b0b-f3e616e484cc None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:44.808722 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5f79ec-6cc6-463c-a9f2-b50b0b15a6ad None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:44.809234 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332c4164-5734-4b47-9b0b-f3e616e484cc None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:44.809732 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5f79ec-6cc6-463c-a9f2-b50b0b15a6ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:44.810845 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332c4164-5734-4b47-9b0b-f3e616e484cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:44.811864 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5f79ec-6cc6-463c-a9f2-b50b0b15a6ad None admin] RBAC: Adding query filter params (role.id=4559b9a0837249f895a13c70096f8d38, scope.project.id=730685c4f62b43c493d42b5897c99a02, user.id=ca5d3774b93a41d1a8b95796b7899d4c) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:19:44.812375 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5f79ec-6cc6-463c-a9f2-b50b0b15a6ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:44.813392 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332c4164-5734-4b47-9b0b-f3e616e484cc None admin] RBAC: Adding query filter params (role.id=6d8b63d350944d78812edbefb32ae8ee, scope.project.id=730685c4f62b43c493d42b5897c99a02, user.id=1cf42c5c30ba4b9a93fc4cdb9713c6cc) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:19:44.815057 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5f79ec-6cc6-463c-a9f2-b50b0b15a6ad None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:44.815573 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332c4164-5734-4b47-9b0b-f3e616e484cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:44.817236 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332c4164-5734-4b47-9b0b-f3e616e484cc None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:44.827663 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 51/94] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Aug 6 01:19:44 2022] GET /identity/v3/role_assignments?user.id=1cf42c5c30ba4b9a93fc4cdb9713c6cc&scope.project.id=730685c4f62b43c493d42b5897c99a02&role.id=6d8b63d350944d78812edbefb32ae8ee => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:19:44.830996 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 44/95] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Aug 6 01:19:44 2022] GET /identity/v3/role_assignments?user.id=ca5d3774b93a41d1a8b95796b7899d4c&scope.project.id=730685c4f62b43c493d42b5897c99a02&role.id=4559b9a0837249f895a13c70096f8d38 => generated 259 bytes in 403 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:19:44.832458 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332c4164-5734-4b47-9b0b-f3e616e484cc None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:44.956851 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332c4164-5734-4b47-9b0b-f3e616e484cc None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:44.999970 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df552dc-b718-497a-8bf5-47a69daec293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:45.000646 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df552dc-b718-497a-8bf5-47a69daec293 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:45.000935 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df552dc-b718-497a-8bf5-47a69daec293 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:45.001265 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df552dc-b718-497a-8bf5-47a69daec293 None admin] PATH_INFO: `/v3/projects/3d00a20dc2ed44be99579eaac1c6dfa1` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:45.002504 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df552dc-b718-497a-8bf5-47a69daec293 None admin] RBAC: Authorizing `identity:get_project(project_id=3d00a20dc2ed44be99579eaac1c6dfa1)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:45.003979 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df552dc-b718-497a-8bf5-47a69daec293 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:45.005912 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 52/96] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Aug 6 01:19:44 2022] GET /identity/v3/projects/3d00a20dc2ed44be99579eaac1c6dfa1 => generated 312 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:19:45.011941 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5f79ec-6cc6-463c-a9f2-b50b0b15a6ad None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:45.157458 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5f79ec-6cc6-463c-a9f2-b50b0b15a6ad None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:45.184992 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:45.194327 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5dd917-75a8-4e68-99a6-51a98b163239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:45.194964 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5dd917-75a8-4e68-99a6-51a98b163239 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:45.195252 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5dd917-75a8-4e68-99a6-51a98b163239 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:45.195591 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5dd917-75a8-4e68-99a6-51a98b163239 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:45.196404 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5dd917-75a8-4e68-99a6-51a98b163239 None admin] RBAC: Adding query filter params (role.id=61b3b93d37a047e4bb8369b46fe20db9, scope.project.id=3d00a20dc2ed44be99579eaac1c6dfa1, user.id=1cf42c5c30ba4b9a93fc4cdb9713c6cc) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:19:45.196723 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5dd917-75a8-4e68-99a6-51a98b163239 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:45.198275 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5dd917-75a8-4e68-99a6-51a98b163239 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:45.205229 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 45/97] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Aug 6 01:19:45 2022] GET /identity/v3/role_assignments?user.id=1cf42c5c30ba4b9a93fc4cdb9713c6cc&scope.project.id=3d00a20dc2ed44be99579eaac1c6dfa1&role.id=61b3b93d37a047e4bb8369b46fe20db9 => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:19:46.417375 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee36b63b-cc30-49b6-86e9-42150b824c4d None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:46.417973 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee36b63b-cc30-49b6-86e9-42150b824c4d None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:46.417973 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee36b63b-cc30-49b6-86e9-42150b824c4d None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:46.419307 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 53/98] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:19:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:19:46.424871 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c7b8da-40c5-4770-9b57-0615323bfad9 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:46.425364 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c7b8da-40c5-4770-9b57-0615323bfad9 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:46.425598 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c7b8da-40c5-4770-9b57-0615323bfad9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:46.444005 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-42c7b8da-40c5-4770-9b57-0615323bfad9 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:19:46.445687 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-42c7b8da-40c5-4770-9b57-0615323bfad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:19:46.530162 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 46/99] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:19:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:19:46.553840 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee36b63b-cc30-49b6-86e9-42150b824c4d None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:46.599685 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:46.656879 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e321ce4-7f87-47d8-9d07-756edff04a26 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:46.657539 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e321ce4-7f87-47d8-9d07-756edff04a26 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:46.658034 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e321ce4-7f87-47d8-9d07-756edff04a26 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:46.659591 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 47/100] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:19:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 01:19:46.664872 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796551f3-c140-491c-9b9f-fab141caef18 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:46.665434 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796551f3-c140-491c-9b9f-fab141caef18 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:46.665908 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796551f3-c140-491c-9b9f-fab141caef18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:46.689793 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee36b63b-cc30-49b6-86e9-42150b824c4d None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:46.695158 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-796551f3-c140-491c-9b9f-fab141caef18 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:19:46.696612 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-796551f3-c140-491c-9b9f-fab141caef18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:19:46.754418 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5d7b77-9af2-47e3-b19f-74c7e7b3eccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:46.755429 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5d7b77-9af2-47e3-b19f-74c7e7b3eccd None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:46.755935 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5d7b77-9af2-47e3-b19f-74c7e7b3eccd None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:46.756443 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5d7b77-9af2-47e3-b19f-74c7e7b3eccd None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:46.768237 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5d7b77-9af2-47e3-b19f-74c7e7b3eccd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:19:46.774199 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-ae5d7b77-9af2-47e3-b19f-74c7e7b3eccd None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 06 01:19:46.774199 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:19:46.774199 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:19:46.774199 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:19:46.774199 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:19:46.774199 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:19:46.774199 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:19:46.774199 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:19:46.774199 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:19:46.774199 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:19:46.774199 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:19:46.774199 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:19:46.774199 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:19:46.774199 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:19:46.774199 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:19:46.774199 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:19:46.775068 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:19:46.775068 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:19:46.775068 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:19:46.775068 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:19:46.775068 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:19:46.775068 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:19:46.775068 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:19:46.775068 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:19:46.775068 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:19:46.775068 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:19:46.775068 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:19:46.775068 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:19:46.775068 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:19:46.775068 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:19:46.775068 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:19:46.775068 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:19:46.775846 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:19:46.775846 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:19:46.775846 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:19:46.775846 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:19:46.775846 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:19:46.775846 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:19:46.775846 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:19:46.775846 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:19:46.775846 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:19:46.775846 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:19:46.775846 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 06 01:19:46.775846 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:19:46.778339 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 54/101] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Aug 6 01:19:46 2022] GET /identity/v3/roles/reader => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:19:46.784348 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5d7b77-9af2-47e3-b19f-74c7e7b3eccd None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:46.850971 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 48/102] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:19:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 01:19:46.886119 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796551f3-c140-491c-9b9f-fab141caef18 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:46.913550 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5d7b77-9af2-47e3-b19f-74c7e7b3eccd None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:46.921461 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:46.950549 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41da6d91-fd55-4700-8220-8c74bfa6e2b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:46.952284 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41da6d91-fd55-4700-8220-8c74bfa6e2b1 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:46.952771 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41da6d91-fd55-4700-8220-8c74bfa6e2b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:46.953319 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41da6d91-fd55-4700-8220-8c74bfa6e2b1 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:46.954238 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41da6d91-fd55-4700-8220-8c74bfa6e2b1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:19:46.954777 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41da6d91-fd55-4700-8220-8c74bfa6e2b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:46.956425 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41da6d91-fd55-4700-8220-8c74bfa6e2b1 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:46.966377 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 55/103] 10.222.0.26 () {56 vars in 1101 bytes} [Sat Aug 6 01:19:46 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:19:46.974288 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41da6d91-fd55-4700-8220-8c74bfa6e2b1 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:46.997360 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:47.101434 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796551f3-c140-491c-9b9f-fab141caef18 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:47.179814 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0281bafc-4acc-4721-b140-3f369efa7dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:47.180683 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0281bafc-4acc-4721-b140-3f369efa7dcd None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:47.180969 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0281bafc-4acc-4721-b140-3f369efa7dcd None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:47.181279 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0281bafc-4acc-4721-b140-3f369efa7dcd None admin] PATH_INFO: `/v3/domains/default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:47.183331 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0281bafc-4acc-4721-b140-3f369efa7dcd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:47.184807 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0281bafc-4acc-4721-b140-3f369efa7dcd None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:47.186849 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 49/104] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:19:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:19:47.194678 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0281bafc-4acc-4721-b140-3f369efa7dcd None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:47.201483 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41da6d91-fd55-4700-8220-8c74bfa6e2b1 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:47.273033 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd71898b-7a4d-431b-ba38-0d94a19197dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:47.273712 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd71898b-7a4d-431b-ba38-0d94a19197dc None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:47.274000 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd71898b-7a4d-431b-ba38-0d94a19197dc None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:47.274280 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd71898b-7a4d-431b-ba38-0d94a19197dc None admin] PATH_INFO: `/v3/users/3b686ca118134310a61f9e8157e9b9be` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:47.302212 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd71898b-7a4d-431b-ba38-0d94a19197dc None admin] RBAC: Authorizing `identity:get_user(user_id=3b686ca118134310a61f9e8157e9b9be)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:47.303566 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd71898b-7a4d-431b-ba38-0d94a19197dc None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:47.305327 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 56/105] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Aug 6 01:19:46 2022] GET /identity/v3/users/3b686ca118134310a61f9e8157e9b9be => generated 289 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:19:47.311485 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd71898b-7a4d-431b-ba38-0d94a19197dc None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:47.348602 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:47.392653 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0281bafc-4acc-4721-b140-3f369efa7dcd None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:47.425422 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:47.457750 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741476de-99af-4c79-bbfe-36b72dd1d830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:47.458391 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741476de-99af-4c79-bbfe-36b72dd1d830 None admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:47.458677 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741476de-99af-4c79-bbfe-36b72dd1d830 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:47.458955 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741476de-99af-4c79-bbfe-36b72dd1d830 None admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:47.459655 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741476de-99af-4c79-bbfe-36b72dd1d830 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:47.461131 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741476de-99af-4c79-bbfe-36b72dd1d830 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:47.518323 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 50/106] 10.222.0.26 () {60 vars in 1141 bytes} [Sat Aug 6 01:19:47 2022] POST /identity/v3/projects => generated 302 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:19:47.522227 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd71898b-7a4d-431b-ba38-0d94a19197dc None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:47.570833 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee54a2e-8609-4934-af8e-7cfe86cba126 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:47.571128 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee54a2e-8609-4934-af8e-7cfe86cba126 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:47.571403 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee54a2e-8609-4934-af8e-7cfe86cba126 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:47.572897 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 51/107] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:19:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:19:47.578539 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cabce1a-bf4b-493f-845d-0ee34dc2b322 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:47.578916 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cabce1a-bf4b-493f-845d-0ee34dc2b322 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:47.579119 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cabce1a-bf4b-493f-845d-0ee34dc2b322 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:47.582212 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee067cd-ac9b-40e8-8ff5-b76b28997a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:47.582865 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee067cd-ac9b-40e8-8ff5-b76b28997a34 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:47.583169 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee067cd-ac9b-40e8-8ff5-b76b28997a34 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:47.583535 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee067cd-ac9b-40e8-8ff5-b76b28997a34 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:47.614080 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-2cabce1a-bf4b-493f-845d-0ee34dc2b322 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:19:47.615301 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-2cabce1a-bf4b-493f-845d-0ee34dc2b322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:19:47.668031 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee067cd-ac9b-40e8-8ff5-b76b28997a34 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:47.670556 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee067cd-ac9b-40e8-8ff5-b76b28997a34 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:47.705955 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:47.803171 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 52/108] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:19:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:19:47.804230 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 57/109] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Aug 6 01:19:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 493 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 01:19:47.804667 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623124ea-0ee3-4cff-84ed-a01d278b69a7 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:47.809076 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623124ea-0ee3-4cff-84ed-a01d278b69a7 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:47.809076 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623124ea-0ee3-4cff-84ed-a01d278b69a7 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:47.810880 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee067cd-ac9b-40e8-8ff5-b76b28997a34 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:47.813237 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 53/110] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:19:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:19:47.814630 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1cec19-46d9-46e0-81f6-4e42b56f9229 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:47.814925 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1cec19-46d9-46e0-81f6-4e42b56f9229 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:47.815206 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1cec19-46d9-46e0-81f6-4e42b56f9229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:47.837678 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-5f1cec19-46d9-46e0-81f6-4e42b56f9229 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:19:47.839060 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-5f1cec19-46d9-46e0-81f6-4e42b56f9229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:19:47.921335 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 54/111] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:19:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:19:47.922694 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd4b91a-84b5-4d0b-9d13-cd9d70e0e4a2 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:47.922993 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd4b91a-84b5-4d0b-9d13-cd9d70e0e4a2 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:47.923275 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd4b91a-84b5-4d0b-9d13-cd9d70e0e4a2 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:47.924603 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 55/112] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:19:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:19:47.925333 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd4b91a-84b5-4d0b-9d13-cd9d70e0e4a2 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:47.947166 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee067cd-ac9b-40e8-8ff5-b76b28997a34 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:47.970696 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:47.980032 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:47.996536 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85162e8e-761d-4492-8ed8-782a2c8e44f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:47.998328 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85162e8e-761d-4492-8ed8-782a2c8e44f6 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:47.999206 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85162e8e-761d-4492-8ed8-782a2c8e44f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:47.999581 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85162e8e-761d-4492-8ed8-782a2c8e44f6 None admin] PATH_INFO: `/v3/projects/730685c4f62b43c493d42b5897c99a02` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:48.001109 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85162e8e-761d-4492-8ed8-782a2c8e44f6 None admin] RBAC: Authorizing `identity:get_project(project_id=730685c4f62b43c493d42b5897c99a02)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:48.003395 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85162e8e-761d-4492-8ed8-782a2c8e44f6 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:48.005694 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 58/113] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Aug 6 01:19:47 2022] GET /identity/v3/projects/730685c4f62b43c493d42b5897c99a02 => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:19:48.007280 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c32e60-d0d6-46af-8bd9-b9ac726d8dbd None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:48.007571 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c32e60-d0d6-46af-8bd9-b9ac726d8dbd None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:48.007991 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c32e60-d0d6-46af-8bd9-b9ac726d8dbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:48.032624 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-01c32e60-d0d6-46af-8bd9-b9ac726d8dbd None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:19:48.033884 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-01c32e60-d0d6-46af-8bd9-b9ac726d8dbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:19:48.067065 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd4b91a-84b5-4d0b-9d13-cd9d70e0e4a2 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:48.114937 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c05faa-81f0-40fc-bf3a-f00b9a6b5f52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:48.115589 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c05faa-81f0-40fc-bf3a-f00b9a6b5f52 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:48.115876 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c05faa-81f0-40fc-bf3a-f00b9a6b5f52 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:48.116157 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c05faa-81f0-40fc-bf3a-f00b9a6b5f52 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:48.134289 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c05faa-81f0-40fc-bf3a-f00b9a6b5f52 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:19:48.134651 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 59/114] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:19:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:19:48.135503 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c32e60-d0d6-46af-8bd9-b9ac726d8dbd None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:48.141886 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-13c05faa-81f0-40fc-bf3a-f00b9a6b5f52 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 01:19:48.141886 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:19:48.141886 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:19:48.141886 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:19:48.141886 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:19:48.141886 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:19:48.141886 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:19:48.141886 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:19:48.141886 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:19:48.141886 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:19:48.141886 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:19:48.141886 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:19:48.141886 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:19:48.141886 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:19:48.141886 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:19:48.141886 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:19:48.142719 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:19:48.142719 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:19:48.142719 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:19:48.142719 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:19:48.142719 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:19:48.142719 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:19:48.142719 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:19:48.142719 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:19:48.142719 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:19:48.142719 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:19:48.142719 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:19:48.142719 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:19:48.142719 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:19:48.142719 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:19:48.142719 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:19:48.142719 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:19:48.143516 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:19:48.143516 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:19:48.143516 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:19:48.143516 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:19:48.143516 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:19:48.143516 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:19:48.143516 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:19:48.143516 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:19:48.143516 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:19:48.143516 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:19:48.143516 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 01:19:48.143516 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:19:48.145999 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 56/115] 10.222.0.26 () {56 vars in 1114 bytes} [Sat Aug 6 01:19:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:19:48.147333 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c05faa-81f0-40fc-bf3a-f00b9a6b5f52 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:48.252626 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:48.262539 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c05faa-81f0-40fc-bf3a-f00b9a6b5f52 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:48.273157 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c32e60-d0d6-46af-8bd9-b9ac726d8dbd None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:48.291174 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:48.300575 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9943fb5-a5fd-4141-a80d-3cc8bab3e591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:48.301230 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9943fb5-a5fd-4141-a80d-3cc8bab3e591 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:48.301612 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9943fb5-a5fd-4141-a80d-3cc8bab3e591 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:48.301895 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9943fb5-a5fd-4141-a80d-3cc8bab3e591 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:48.302697 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9943fb5-a5fd-4141-a80d-3cc8bab3e591 None admin] RBAC: Adding query filter params (role.id=25efb2aed94e4d63b0b6cb8faf5a2ee7, scope.project.id=730685c4f62b43c493d42b5897c99a02, user.id=3b686ca118134310a61f9e8157e9b9be) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:19:48.303226 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9943fb5-a5fd-4141-a80d-3cc8bab3e591 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:48.304460 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9943fb5-a5fd-4141-a80d-3cc8bab3e591 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:48.311186 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 57/116] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Aug 6 01:19:48 2022] GET /identity/v3/role_assignments?user.id=3b686ca118134310a61f9e8157e9b9be&scope.project.id=730685c4f62b43c493d42b5897c99a02&role.id=25efb2aed94e4d63b0b6cb8faf5a2ee7 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:19:48.311830 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9943fb5-a5fd-4141-a80d-3cc8bab3e591 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:48.312129 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7187d4-cc3c-42a4-b84d-74295a9c9089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:48.312790 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7187d4-cc3c-42a4-b84d-74295a9c9089 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:48.317379 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7187d4-cc3c-42a4-b84d-74295a9c9089 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:48.317379 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7187d4-cc3c-42a4-b84d-74295a9c9089 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:48.336079 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7187d4-cc3c-42a4-b84d-74295a9c9089 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:19:48.344418 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-3e7187d4-cc3c-42a4-b84d-74295a9c9089 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 06 01:19:48.344418 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:19:48.344418 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:19:48.344418 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:19:48.344418 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:19:48.344418 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:19:48.344418 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:19:48.344418 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:19:48.344418 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:19:48.344418 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:19:48.344418 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:19:48.344418 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:19:48.344418 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:19:48.344418 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:19:48.344418 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:19:48.344418 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:19:48.345389 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:19:48.345389 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:19:48.345389 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:19:48.345389 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:19:48.345389 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:19:48.345389 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:19:48.345389 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:19:48.345389 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:19:48.345389 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:19:48.345389 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:19:48.345389 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:19:48.345389 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:19:48.345389 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:19:48.345389 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:19:48.345389 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:19:48.345389 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:19:48.346139 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:19:48.346139 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:19:48.346139 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:19:48.346139 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:19:48.346139 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:19:48.346139 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:19:48.346139 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:19:48.346139 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:19:48.346139 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:19:48.346139 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:19:48.346139 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 06 01:19:48.346139 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:19:48.346710 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 60/117] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Aug 6 01:19:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:19:48.349214 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7187d4-cc3c-42a4-b84d-74295a9c9089 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:48.463068 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9943fb5-a5fd-4141-a80d-3cc8bab3e591 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:48.487604 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7187d4-cc3c-42a4-b84d-74295a9c9089 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:48.500772 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e6cef6-7e84-4a49-822d-5184c03b5de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:48.501574 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e6cef6-7e84-4a49-822d-5184c03b5de4 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:48.502044 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e6cef6-7e84-4a49-822d-5184c03b5de4 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:48.502435 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e6cef6-7e84-4a49-822d-5184c03b5de4 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:48.503302 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e6cef6-7e84-4a49-822d-5184c03b5de4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:19:48.503737 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e6cef6-7e84-4a49-822d-5184c03b5de4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:48.505290 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e6cef6-7e84-4a49-822d-5184c03b5de4 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:48.518690 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 58/118] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Aug 6 01:19:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:19:48.520026 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af338a40-f22c-4d78-beb7-624fc6de2bbf None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:48.520427 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af338a40-f22c-4d78-beb7-624fc6de2bbf None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:48.520810 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af338a40-f22c-4d78-beb7-624fc6de2bbf None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:48.522454 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 59/119] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:19:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:19:48.523275 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af338a40-f22c-4d78-beb7-624fc6de2bbf None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:48.531235 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e3a45d-efc8-4755-b096-3d3a257641b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:48.532403 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e3a45d-efc8-4755-b096-3d3a257641b7 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:48.532836 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e3a45d-efc8-4755-b096-3d3a257641b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:48.533325 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e3a45d-efc8-4755-b096-3d3a257641b7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:48.557073 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:48.557244 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e3a45d-efc8-4755-b096-3d3a257641b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:19:48.559456 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-69e3a45d-efc8-4755-b096-3d3a257641b7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:19:48.559456 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:19:48.559456 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:19:48.559456 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:19:48.559456 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:19:48.559456 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:19:48.559456 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:19:48.559456 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:19:48.559456 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:19:48.559456 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:19:48.559456 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:19:48.559456 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:19:48.559456 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:19:48.559456 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:19:48.559456 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:19:48.559456 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:19:48.560464 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:19:48.560464 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:19:48.560464 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:19:48.560464 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:19:48.560464 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:19:48.560464 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:19:48.560464 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:19:48.560464 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:19:48.560464 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:19:48.560464 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:19:48.560464 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:19:48.560464 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:19:48.560464 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:19:48.560464 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:19:48.560464 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:19:48.560464 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:19:48.561590 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:19:48.561590 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:19:48.561590 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:19:48.561590 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:19:48.561590 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:19:48.561590 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:19:48.561590 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:19:48.561590 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:19:48.561590 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:19:48.561590 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:19:48.561590 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:19:48.561590 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:19:48.561590 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 61/120] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Aug 6 01:19:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:19:48.566205 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e3a45d-efc8-4755-b096-3d3a257641b7 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:48.605908 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:48.639863 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af338a40-f22c-4d78-beb7-624fc6de2bbf None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:48.677225 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4279ff24-5c39-496d-ba3c-156088505d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:48.678065 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4279ff24-5c39-496d-ba3c-156088505d48 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:48.678423 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4279ff24-5c39-496d-ba3c-156088505d48 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:48.678773 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4279ff24-5c39-496d-ba3c-156088505d48 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:48.679620 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4279ff24-5c39-496d-ba3c-156088505d48 None admin] RBAC: Adding query filter params (name=member) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:19:48.679999 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4279ff24-5c39-496d-ba3c-156088505d48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:48.680709 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e3a45d-efc8-4755-b096-3d3a257641b7 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:48.681721 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4279ff24-5c39-496d-ba3c-156088505d48 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:48.696630 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 60/121] 10.222.0.26 () {56 vars in 1101 bytes} [Sat Aug 6 01:19:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:19:48.698052 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660afa37-5d32-4a26-ba54-a1741b987874 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:48.698417 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660afa37-5d32-4a26-ba54-a1741b987874 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:48.698816 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660afa37-5d32-4a26-ba54-a1741b987874 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:48.718550 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9f0e80-8954-4647-94d8-20145d8ac269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:48.718968 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-660afa37-5d32-4a26-ba54-a1741b987874 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:19:48.719200 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9f0e80-8954-4647-94d8-20145d8ac269 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:48.719524 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9f0e80-8954-4647-94d8-20145d8ac269 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:48.719816 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9f0e80-8954-4647-94d8-20145d8ac269 None admin] PATH_INFO: `/v3/users/1cf42c5c30ba4b9a93fc4cdb9713c6cc` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:48.720458 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-660afa37-5d32-4a26-ba54-a1741b987874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:19:48.721012 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9f0e80-8954-4647-94d8-20145d8ac269 None admin] RBAC: Authorizing `identity:get_user(user_id=1cf42c5c30ba4b9a93fc4cdb9713c6cc)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:48.722594 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9f0e80-8954-4647-94d8-20145d8ac269 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:48.724452 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 62/122] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Aug 6 01:19:48 2022] GET /identity/v3/users/1cf42c5c30ba4b9a93fc4cdb9713c6cc => generated 275 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:19:48.725250 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9f0e80-8954-4647-94d8-20145d8ac269 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:48.798563 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 61/123] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:19:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:19:48.799284 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660afa37-5d32-4a26-ba54-a1741b987874 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:48.812288 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:48.868034 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9f0e80-8954-4647-94d8-20145d8ac269 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:48.871576 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:48.917315 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dee28e7-0437-4fa6-931c-cc10b8247e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:48.918391 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dee28e7-0437-4fa6-931c-cc10b8247e66 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:48.918826 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dee28e7-0437-4fa6-931c-cc10b8247e66 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:48.919252 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dee28e7-0437-4fa6-931c-cc10b8247e66 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:48.920374 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660afa37-5d32-4a26-ba54-a1741b987874 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:48.920825 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dee28e7-0437-4fa6-931c-cc10b8247e66 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:19:48.921315 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dee28e7-0437-4fa6-931c-cc10b8247e66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:48.923828 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dee28e7-0437-4fa6-931c-cc10b8247e66 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:48.933941 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 63/124] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Aug 6 01:19:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:19:48.935827 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dee28e7-0437-4fa6-931c-cc10b8247e66 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:48.960000 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ec35c4-b73b-4d03-ae2e-5b0cdd4a8bbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:48.960820 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ec35c4-b73b-4d03-ae2e-5b0cdd4a8bbc None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:48.961252 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ec35c4-b73b-4d03-ae2e-5b0cdd4a8bbc None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:48.961668 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ec35c4-b73b-4d03-ae2e-5b0cdd4a8bbc None admin] PATH_INFO: `/v3/users/1cf42c5c30ba4b9a93fc4cdb9713c6cc` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:48.963524 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ec35c4-b73b-4d03-ae2e-5b0cdd4a8bbc None admin] RBAC: Authorizing `identity:get_user(user_id=1cf42c5c30ba4b9a93fc4cdb9713c6cc)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:48.965528 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ec35c4-b73b-4d03-ae2e-5b0cdd4a8bbc None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:48.968470 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 62/125] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Aug 6 01:19:48 2022] GET /identity/v3/users/1cf42c5c30ba4b9a93fc4cdb9713c6cc => generated 275 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:19:48.973510 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ec35c4-b73b-4d03-ae2e-5b0cdd4a8bbc None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:49.072928 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dee28e7-0437-4fa6-931c-cc10b8247e66 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:49.079123 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:49.107794 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ec35c4-b73b-4d03-ae2e-5b0cdd4a8bbc None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:49.122118 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24ee0c2-e9e7-4350-895e-dd530933d717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:49.123977 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24ee0c2-e9e7-4350-895e-dd530933d717 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:49.124410 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24ee0c2-e9e7-4350-895e-dd530933d717 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:49.124998 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24ee0c2-e9e7-4350-895e-dd530933d717 None admin] PATH_INFO: `/v3/projects/730685c4f62b43c493d42b5897c99a02` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:49.126811 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24ee0c2-e9e7-4350-895e-dd530933d717 None admin] RBAC: Authorizing `identity:get_project(project_id=730685c4f62b43c493d42b5897c99a02)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:49.128693 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24ee0c2-e9e7-4350-895e-dd530933d717 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:49.130839 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 64/126] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Aug 6 01:19:48 2022] GET /identity/v3/projects/730685c4f62b43c493d42b5897c99a02 => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:19:49.131791 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24ee0c2-e9e7-4350-895e-dd530933d717 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:49.153352 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a8f78a-0851-4d6f-bc9f-8bdecf5be322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:49.154905 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a8f78a-0851-4d6f-bc9f-8bdecf5be322 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:49.155244 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a8f78a-0851-4d6f-bc9f-8bdecf5be322 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:49.155529 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a8f78a-0851-4d6f-bc9f-8bdecf5be322 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:49.169579 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a8f78a-0851-4d6f-bc9f-8bdecf5be322 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:19:49.174007 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:49.178245 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-93a8f78a-0851-4d6f-bc9f-8bdecf5be322 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 06 01:19:49.178245 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:19:49.178245 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:19:49.178245 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:19:49.178245 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:19:49.178245 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:19:49.178245 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:19:49.178245 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:19:49.178245 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:19:49.178245 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:19:49.178245 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:19:49.178245 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:19:49.178245 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:19:49.178245 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:19:49.178245 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:19:49.178245 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:19:49.179119 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:19:49.179119 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:19:49.179119 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:19:49.179119 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:19:49.179119 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:19:49.179119 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:19:49.179119 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:19:49.179119 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:19:49.179119 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:19:49.179119 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:19:49.179119 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:19:49.179119 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:19:49.179119 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:19:49.179119 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:19:49.179119 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:19:49.179119 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:19:49.179887 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:19:49.179887 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:19:49.179887 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:19:49.179887 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:19:49.179887 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:19:49.179887 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:19:49.179887 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:19:49.179887 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:19:49.179887 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:19:49.179887 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:19:49.179887 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 06 01:19:49.179887 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:19:49.179887 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 63/127] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Aug 6 01:19:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:19:49.180503 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a8f78a-0851-4d6f-bc9f-8bdecf5be322 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:49.272608 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24ee0c2-e9e7-4350-895e-dd530933d717 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:49.311606 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd4ddeb-4b58-4c97-b619-90e64263ada0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:49.312192 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a8f78a-0851-4d6f-bc9f-8bdecf5be322 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:49.312310 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd4ddeb-4b58-4c97-b619-90e64263ada0 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:49.312528 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd4ddeb-4b58-4c97-b619-90e64263ada0 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:49.312805 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd4ddeb-4b58-4c97-b619-90e64263ada0 None admin] PATH_INFO: `/v3/users/ca5d3774b93a41d1a8b95796b7899d4c` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:49.314019 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd4ddeb-4b58-4c97-b619-90e64263ada0 None admin] RBAC: Authorizing `identity:get_user(user_id=ca5d3774b93a41d1a8b95796b7899d4c)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:49.315967 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd4ddeb-4b58-4c97-b619-90e64263ada0 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:49.317292 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 65/128] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Aug 6 01:19:49 2022] GET /identity/v3/users/ca5d3774b93a41d1a8b95796b7899d4c => generated 247 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:19:49.317871 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd4ddeb-4b58-4c97-b619-90e64263ada0 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:49.352464 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8be05f3-48d1-41c8-86d4-2783445a93ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:49.353630 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8be05f3-48d1-41c8-86d4-2783445a93ef None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:49.354131 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8be05f3-48d1-41c8-86d4-2783445a93ef None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:49.354555 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8be05f3-48d1-41c8-86d4-2783445a93ef None admin] PATH_INFO: `/v3/projects/730685c4f62b43c493d42b5897c99a02` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:49.356128 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8be05f3-48d1-41c8-86d4-2783445a93ef None admin] RBAC: Authorizing `identity:get_project(project_id=730685c4f62b43c493d42b5897c99a02)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:49.358348 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8be05f3-48d1-41c8-86d4-2783445a93ef None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:49.360402 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 64/129] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Aug 6 01:19:49 2022] GET /identity/v3/projects/730685c4f62b43c493d42b5897c99a02 => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:19:49.361423 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8be05f3-48d1-41c8-86d4-2783445a93ef None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:49.370466 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:49.430160 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd4ddeb-4b58-4c97-b619-90e64263ada0 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:49.431744 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:49.468298 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331f2d71-af16-4eaa-8fda-746ab87dd166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:49.469226 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331f2d71-af16-4eaa-8fda-746ab87dd166 None admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:49.469428 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331f2d71-af16-4eaa-8fda-746ab87dd166 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:49.469712 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331f2d71-af16-4eaa-8fda-746ab87dd166 None admin] PATH_INFO: `/v3/projects/730685c4f62b43c493d42b5897c99a02/users/1cf42c5c30ba4b9a93fc4cdb9713c6cc/roles/6d8b63d350944d78812edbefb32ae8ee` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:49.472225 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331f2d71-af16-4eaa-8fda-746ab87dd166 None admin] RBAC: Authorizing `identity:create_grant(project_id=730685c4f62b43c493d42b5897c99a02, user_id=1cf42c5c30ba4b9a93fc4cdb9713c6cc, role_id=6d8b63d350944d78812edbefb32ae8ee)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:49.473969 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331f2d71-af16-4eaa-8fda-746ab87dd166 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:49.480569 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8be05f3-48d1-41c8-86d4-2783445a93ef None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:49.490106 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 66/130] 10.222.0.26 () {58 vars in 1439 bytes} [Sat Aug 6 01:19:49 2022] PUT /identity/v3/projects/730685c4f62b43c493d42b5897c99a02/users/1cf42c5c30ba4b9a93fc4cdb9713c6cc/roles/6d8b63d350944d78812edbefb32ae8ee => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:19:49.491347 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331f2d71-af16-4eaa-8fda-746ab87dd166 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:49.520285 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814fafd3-446f-4ad2-b6c1-865179aada38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:49.521171 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814fafd3-446f-4ad2-b6c1-865179aada38 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:49.521569 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814fafd3-446f-4ad2-b6c1-865179aada38 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:49.521974 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814fafd3-446f-4ad2-b6c1-865179aada38 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:49.522940 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814fafd3-446f-4ad2-b6c1-865179aada38 None admin] RBAC: Adding query filter params (name=member) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:19:49.523363 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814fafd3-446f-4ad2-b6c1-865179aada38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:49.525119 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814fafd3-446f-4ad2-b6c1-865179aada38 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:49.535272 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 65/131] 10.222.0.26 () {56 vars in 1101 bytes} [Sat Aug 6 01:19:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:19:49.536833 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814fafd3-446f-4ad2-b6c1-865179aada38 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:49.611921 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331f2d71-af16-4eaa-8fda-746ab87dd166 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:49.616928 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:49.655096 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950a8c95-e546-48fd-a894-c07061dc25d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:49.656019 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950a8c95-e546-48fd-a894-c07061dc25d5 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:49.656327 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814fafd3-446f-4ad2-b6c1-865179aada38 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:49.656678 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950a8c95-e546-48fd-a894-c07061dc25d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:49.657150 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950a8c95-e546-48fd-a894-c07061dc25d5 None admin] PATH_INFO: `/v3/projects/730685c4f62b43c493d42b5897c99a02` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:49.658701 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950a8c95-e546-48fd-a894-c07061dc25d5 None admin] RBAC: Authorizing `identity:get_project(project_id=730685c4f62b43c493d42b5897c99a02)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:49.660633 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950a8c95-e546-48fd-a894-c07061dc25d5 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:49.662762 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 67/132] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Aug 6 01:19:49 2022] GET /identity/v3/projects/730685c4f62b43c493d42b5897c99a02 => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:19:49.663731 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950a8c95-e546-48fd-a894-c07061dc25d5 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:49.695396 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee7d6e7-1ae7-4903-a389-4598eafc3618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:49.696229 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee7d6e7-1ae7-4903-a389-4598eafc3618 None admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:49.696594 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee7d6e7-1ae7-4903-a389-4598eafc3618 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:49.696967 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee7d6e7-1ae7-4903-a389-4598eafc3618 None admin] PATH_INFO: `/v3/projects/730685c4f62b43c493d42b5897c99a02/users/1cf42c5c30ba4b9a93fc4cdb9713c6cc/roles/61b3b93d37a047e4bb8369b46fe20db9` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:49.699274 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee7d6e7-1ae7-4903-a389-4598eafc3618 None admin] RBAC: Authorizing `identity:create_grant(project_id=730685c4f62b43c493d42b5897c99a02, user_id=1cf42c5c30ba4b9a93fc4cdb9713c6cc, role_id=61b3b93d37a047e4bb8369b46fe20db9)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:49.701002 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee7d6e7-1ae7-4903-a389-4598eafc3618 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:49.706125 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:49.722449 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 66/133] 10.222.0.26 () {58 vars in 1439 bytes} [Sat Aug 6 01:19:49 2022] PUT /identity/v3/projects/730685c4f62b43c493d42b5897c99a02/users/1cf42c5c30ba4b9a93fc4cdb9713c6cc/roles/61b3b93d37a047e4bb8369b46fe20db9 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:19:49.723404 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee7d6e7-1ae7-4903-a389-4598eafc3618 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:49.807533 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950a8c95-e546-48fd-a894-c07061dc25d5 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:49.848809 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e46566-0fe8-4593-920d-c5661bc89bcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:49.849605 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e46566-0fe8-4593-920d-c5661bc89bcd None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:49.849967 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e46566-0fe8-4593-920d-c5661bc89bcd None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:49.850323 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e46566-0fe8-4593-920d-c5661bc89bcd None admin] PATH_INFO: `/v3/users/1cf42c5c30ba4b9a93fc4cdb9713c6cc` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:49.852018 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e46566-0fe8-4593-920d-c5661bc89bcd None admin] RBAC: Authorizing `identity:get_user(user_id=1cf42c5c30ba4b9a93fc4cdb9713c6cc)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:49.853529 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e46566-0fe8-4593-920d-c5661bc89bcd None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:49.855800 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 68/134] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Aug 6 01:19:49 2022] GET /identity/v3/users/1cf42c5c30ba4b9a93fc4cdb9713c6cc => generated 275 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:19:49.859121 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee7d6e7-1ae7-4903-a389-4598eafc3618 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:49.861681 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e46566-0fe8-4593-920d-c5661bc89bcd None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:49.899518 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:49.914918 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab285108-07af-4093-8f02-c75584013f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:49.916537 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab285108-07af-4093-8f02-c75584013f4f None admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:49.917293 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab285108-07af-4093-8f02-c75584013f4f None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:49.917758 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab285108-07af-4093-8f02-c75584013f4f None admin] PATH_INFO: `/v3/projects/730685c4f62b43c493d42b5897c99a02/users/ca5d3774b93a41d1a8b95796b7899d4c/roles/4559b9a0837249f895a13c70096f8d38` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:49.920874 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab285108-07af-4093-8f02-c75584013f4f None admin] RBAC: Authorizing `identity:create_grant(project_id=730685c4f62b43c493d42b5897c99a02, user_id=ca5d3774b93a41d1a8b95796b7899d4c, role_id=4559b9a0837249f895a13c70096f8d38)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:49.922844 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab285108-07af-4093-8f02-c75584013f4f None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:49.942426 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 67/135] 10.222.0.26 () {58 vars in 1439 bytes} [Sat Aug 6 01:19:49 2022] PUT /identity/v3/projects/730685c4f62b43c493d42b5897c99a02/users/ca5d3774b93a41d1a8b95796b7899d4c/roles/4559b9a0837249f895a13c70096f8d38 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:19:49.979372 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:49.984057 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e46566-0fe8-4593-920d-c5661bc89bcd None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:50.030433 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3538045-b8ca-4e8d-af7b-b68116d09e6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:50.031122 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3538045-b8ca-4e8d-af7b-b68116d09e6d None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:50.031413 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3538045-b8ca-4e8d-af7b-b68116d09e6d None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:50.031698 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3538045-b8ca-4e8d-af7b-b68116d09e6d None admin] PATH_INFO: `/v3/projects/3d00a20dc2ed44be99579eaac1c6dfa1` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:50.033095 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3538045-b8ca-4e8d-af7b-b68116d09e6d None admin] RBAC: Authorizing `identity:get_project(project_id=3d00a20dc2ed44be99579eaac1c6dfa1)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:50.034604 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3538045-b8ca-4e8d-af7b-b68116d09e6d None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:50.036419 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 69/136] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Aug 6 01:19:49 2022] GET /identity/v3/projects/3d00a20dc2ed44be99579eaac1c6dfa1 => generated 312 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:19:50.042513 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab285108-07af-4093-8f02-c75584013f4f None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:50.094083 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9f0155-0514-465e-a252-f9edb1b5244f None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:50.094706 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9f0155-0514-465e-a252-f9edb1b5244f None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:50.095188 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9f0155-0514-465e-a252-f9edb1b5244f None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:50.096848 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 70/137] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:19:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 01:19:50.102702 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038f1c64-f30f-41de-b676-5f7c8dc71a3f None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:50.103279 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038f1c64-f30f-41de-b676-5f7c8dc71a3f None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:50.103763 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038f1c64-f30f-41de-b676-5f7c8dc71a3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:50.133819 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-038f1c64-f30f-41de-b676-5f7c8dc71a3f None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:19:50.135752 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-038f1c64-f30f-41de-b676-5f7c8dc71a3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:19:50.223969 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab285108-07af-4093-8f02-c75584013f4f None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:50.270616 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddde0ef-4623-486a-9f78-2c366f8d3621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:50.271749 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddde0ef-4623-486a-9f78-2c366f8d3621 None admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:50.272356 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddde0ef-4623-486a-9f78-2c366f8d3621 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:50.272888 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddde0ef-4623-486a-9f78-2c366f8d3621 None admin] PATH_INFO: `/v3/projects/3d00a20dc2ed44be99579eaac1c6dfa1/users/1cf42c5c30ba4b9a93fc4cdb9713c6cc/roles/61b3b93d37a047e4bb8369b46fe20db9` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:50.278715 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddde0ef-4623-486a-9f78-2c366f8d3621 None admin] RBAC: Authorizing `identity:create_grant(project_id=3d00a20dc2ed44be99579eaac1c6dfa1, user_id=1cf42c5c30ba4b9a93fc4cdb9713c6cc, role_id=61b3b93d37a047e4bb8369b46fe20db9)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:50.280513 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddde0ef-4623-486a-9f78-2c366f8d3621 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:50.283402 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 71/138] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:19:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 01:19:50.302950 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 68/139] 10.222.0.26 () {58 vars in 1439 bytes} [Sat Aug 6 01:19:50 2022] PUT /identity/v3/projects/3d00a20dc2ed44be99579eaac1c6dfa1/users/1cf42c5c30ba4b9a93fc4cdb9713c6cc/roles/61b3b93d37a047e4bb8369b46fe20db9 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:19:50.310969 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038f1c64-f30f-41de-b676-5f7c8dc71a3f None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:50.417483 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:50.421212 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038f1c64-f30f-41de-b676-5f7c8dc71a3f None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:50.460444 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e56c9e-3ce6-4e47-8d1e-ccaac4675e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:50.461171 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e56c9e-3ce6-4e47-8d1e-ccaac4675e4f None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:50.461466 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e56c9e-3ce6-4e47-8d1e-ccaac4675e4f None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:50.461749 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e56c9e-3ce6-4e47-8d1e-ccaac4675e4f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:50.463029 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e56c9e-3ce6-4e47-8d1e-ccaac4675e4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:50.464519 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e56c9e-3ce6-4e47-8d1e-ccaac4675e4f None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:50.471483 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 72/140] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:19:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:19:50.472933 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddde0ef-4623-486a-9f78-2c366f8d3621 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:50.543816 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:50.606026 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddde0ef-4623-486a-9f78-2c366f8d3621 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:50.643267 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2263de-f02c-4266-91e8-912f144ace5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:50.643969 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2263de-f02c-4266-91e8-912f144ace5a None admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:50.644260 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2263de-f02c-4266-91e8-912f144ace5a None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:50.644545 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2263de-f02c-4266-91e8-912f144ace5a None admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:50.645340 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2263de-f02c-4266-91e8-912f144ace5a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:50.646799 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2263de-f02c-4266-91e8-912f144ace5a None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:50.686177 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 69/141] 10.222.0.26 () {60 vars in 1132 bytes} [Sat Aug 6 01:19:50 2022] POST /identity/v3/users => generated 283 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:19:50.712838 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538cb5cd-9607-41b4-82ef-d338824ae001 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:50.713402 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538cb5cd-9607-41b4-82ef-d338824ae001 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:50.713402 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538cb5cd-9607-41b4-82ef-d338824ae001 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:50.715291 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 73/142] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:19:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:19:50.721152 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c9c61f-37aa-4eec-a4e5-45f35317600e None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:50.721523 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c9c61f-37aa-4eec-a4e5-45f35317600e None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:50.721813 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c9c61f-37aa-4eec-a4e5-45f35317600e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:50.742867 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-55c9c61f-37aa-4eec-a4e5-45f35317600e None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:19:50.744173 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-55c9c61f-37aa-4eec-a4e5-45f35317600e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:19:50.823104 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 70/143] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:19:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:19:50.851261 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538cb5cd-9607-41b4-82ef-d338824ae001 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:50.974269 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538cb5cd-9607-41b4-82ef-d338824ae001 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:51.011719 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac0e08d-7b70-495f-916f-ebe158f04b46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:51.012375 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac0e08d-7b70-495f-916f-ebe158f04b46 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:51.012713 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac0e08d-7b70-495f-916f-ebe158f04b46 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:51.013024 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac0e08d-7b70-495f-916f-ebe158f04b46 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:51.023983 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac0e08d-7b70-495f-916f-ebe158f04b46 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:19:51.029070 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-aac0e08d-7b70-495f-916f-ebe158f04b46 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 06 01:19:51.029070 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:19:51.029070 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:19:51.029070 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:19:51.029070 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:19:51.029070 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:19:51.029070 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:19:51.029070 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:19:51.029070 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:19:51.029070 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:19:51.029070 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:19:51.029070 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:19:51.029070 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:19:51.029070 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:19:51.029070 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:19:51.029070 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:19:51.029946 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:19:51.029946 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:19:51.029946 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:19:51.029946 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:19:51.029946 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:19:51.029946 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:19:51.029946 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:19:51.029946 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:19:51.029946 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:19:51.029946 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:19:51.029946 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:19:51.029946 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:19:51.029946 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:19:51.029946 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:19:51.029946 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:19:51.029946 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:19:51.030740 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:19:51.030740 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:19:51.030740 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:19:51.030740 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:19:51.030740 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:19:51.030740 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:19:51.030740 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:19:51.030740 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:19:51.030740 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:19:51.030740 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:19:51.030740 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 06 01:19:51.030740 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:19:51.030740 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 74/144] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Aug 6 01:19:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:19:51.037602 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c9c61f-37aa-4eec-a4e5-45f35317600e None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:51.074654 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:51.147243 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c9c61f-37aa-4eec-a4e5-45f35317600e None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:51.183520 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd30fbe-ff48-494e-b71b-8649377821ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:51.184156 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd30fbe-ff48-494e-b71b-8649377821ce None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:51.184447 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd30fbe-ff48-494e-b71b-8649377821ce None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:51.184736 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd30fbe-ff48-494e-b71b-8649377821ce None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:51.186074 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd30fbe-ff48-494e-b71b-8649377821ce None admin] RBAC: Adding query filter params (name=reader) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:19:51.186389 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd30fbe-ff48-494e-b71b-8649377821ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:51.187801 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd30fbe-ff48-494e-b71b-8649377821ce None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:51.196663 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 71/145] 10.222.0.26 () {56 vars in 1101 bytes} [Sat Aug 6 01:19:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:19:51.203159 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac0e08d-7b70-495f-916f-ebe158f04b46 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:51.238095 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:51.334297 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac0e08d-7b70-495f-916f-ebe158f04b46 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:51.380891 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f748707d-697c-4a41-bf29-90c53b0fb04e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:51.382009 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f748707d-697c-4a41-bf29-90c53b0fb04e None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:51.382503 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f748707d-697c-4a41-bf29-90c53b0fb04e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:51.382987 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f748707d-697c-4a41-bf29-90c53b0fb04e None admin] PATH_INFO: `/v3/users/3b686ca118134310a61f9e8157e9b9be` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:51.384509 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f748707d-697c-4a41-bf29-90c53b0fb04e None admin] RBAC: Authorizing `identity:get_user(user_id=3b686ca118134310a61f9e8157e9b9be)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:51.386378 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f748707d-697c-4a41-bf29-90c53b0fb04e None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:51.388971 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 75/146] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Aug 6 01:19:51 2022] GET /identity/v3/users/3b686ca118134310a61f9e8157e9b9be => generated 289 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:19:51.397357 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd30fbe-ff48-494e-b71b-8649377821ce None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:51.545990 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:51.553336 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd30fbe-ff48-494e-b71b-8649377821ce None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:51.591912 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c5bce6-c1ce-4e6e-b643-96f9323b07e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:51.592677 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c5bce6-c1ce-4e6e-b643-96f9323b07e1 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:51.592969 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c5bce6-c1ce-4e6e-b643-96f9323b07e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:51.593294 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c5bce6-c1ce-4e6e-b643-96f9323b07e1 None admin] PATH_INFO: `/v3/projects/730685c4f62b43c493d42b5897c99a02` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:51.594532 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c5bce6-c1ce-4e6e-b643-96f9323b07e1 None admin] RBAC: Authorizing `identity:get_project(project_id=730685c4f62b43c493d42b5897c99a02)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:51.596192 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c5bce6-c1ce-4e6e-b643-96f9323b07e1 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:51.598095 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 72/147] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Aug 6 01:19:51 2022] GET /identity/v3/projects/730685c4f62b43c493d42b5897c99a02 => generated 298 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:19:51.604200 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f748707d-697c-4a41-bf29-90c53b0fb04e None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:51.719171 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:51.731205 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f748707d-697c-4a41-bf29-90c53b0fb04e None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:51.768293 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d265ed6-9c71-4412-8296-0bd451913af2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:51.769039 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d265ed6-9c71-4412-8296-0bd451913af2 None admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:51.769368 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d265ed6-9c71-4412-8296-0bd451913af2 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:51.769652 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d265ed6-9c71-4412-8296-0bd451913af2 None admin] PATH_INFO: `/v3/projects/730685c4f62b43c493d42b5897c99a02/users/3b686ca118134310a61f9e8157e9b9be/roles/25efb2aed94e4d63b0b6cb8faf5a2ee7` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:51.771663 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d265ed6-9c71-4412-8296-0bd451913af2 None admin] RBAC: Authorizing `identity:create_grant(project_id=730685c4f62b43c493d42b5897c99a02, user_id=3b686ca118134310a61f9e8157e9b9be, role_id=25efb2aed94e4d63b0b6cb8faf5a2ee7)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:51.773161 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d265ed6-9c71-4412-8296-0bd451913af2 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:51.792878 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 76/148] 10.222.0.26 () {58 vars in 1439 bytes} [Sat Aug 6 01:19:51 2022] PUT /identity/v3/projects/730685c4f62b43c493d42b5897c99a02/users/3b686ca118134310a61f9e8157e9b9be/roles/25efb2aed94e4d63b0b6cb8faf5a2ee7 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:19:52.677639 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63d81bb-4539-451e-916b-726ae7ce9c82 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:52.678140 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63d81bb-4539-451e-916b-726ae7ce9c82 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:52.678140 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63d81bb-4539-451e-916b-726ae7ce9c82 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:52.679483 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 73/149] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:19:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:19:52.686418 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188ed96a-af6b-4cfe-85e5-c277885cf3e5 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:52.686717 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188ed96a-af6b-4cfe-85e5-c277885cf3e5 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:52.686998 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188ed96a-af6b-4cfe-85e5-c277885cf3e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:52.711097 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-188ed96a-af6b-4cfe-85e5-c277885cf3e5 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:19:52.712316 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-188ed96a-af6b-4cfe-85e5-c277885cf3e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:19:52.819717 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 77/150] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:19:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:19:52.864090 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63d81bb-4539-451e-916b-726ae7ce9c82 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:52.946469 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93d3177-e81d-4a53-a428-869d13324933 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:52.946789 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93d3177-e81d-4a53-a428-869d13324933 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:52.947046 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93d3177-e81d-4a53-a428-869d13324933 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:52.948750 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 78/151] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:19:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:19:52.962761 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc38061a-265b-46e6-aed1-d069d9d1d0e0 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:52.963068 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc38061a-265b-46e6-aed1-d069d9d1d0e0 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:52.963438 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc38061a-265b-46e6-aed1-d069d9d1d0e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:52.988273 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63d81bb-4539-451e-916b-726ae7ce9c82 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:52.997990 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-fc38061a-265b-46e6-aed1-d069d9d1d0e0 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:19:52.999326 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-fc38061a-265b-46e6-aed1-d069d9d1d0e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:19:53.025014 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5f176d-8ec1-4af8-a52d-85ea0045235d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:53.025697 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5f176d-8ec1-4af8-a52d-85ea0045235d None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:53.025985 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5f176d-8ec1-4af8-a52d-85ea0045235d None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:53.026265 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5f176d-8ec1-4af8-a52d-85ea0045235d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:53.039887 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5f176d-8ec1-4af8-a52d-85ea0045235d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:19:53.045131 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-8e5f176d-8ec1-4af8-a52d-85ea0045235d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 01:19:53.045131 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:19:53.045131 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:19:53.045131 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:19:53.045131 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:19:53.045131 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:19:53.045131 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:19:53.045131 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:19:53.045131 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:19:53.045131 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:19:53.045131 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:19:53.045131 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:19:53.045131 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:19:53.045131 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:19:53.045131 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:19:53.045131 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:19:53.047482 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:19:53.047482 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:19:53.047482 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:19:53.047482 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:19:53.047482 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:19:53.047482 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:19:53.047482 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:19:53.047482 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:19:53.047482 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:19:53.047482 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:19:53.047482 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:19:53.047482 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:19:53.047482 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:19:53.047482 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:19:53.047482 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:19:53.047482 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:19:53.049426 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:19:53.049426 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:19:53.049426 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:19:53.049426 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:19:53.049426 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:19:53.049426 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:19:53.049426 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:19:53.049426 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:19:53.049426 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:19:53.049426 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:19:53.049426 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 01:19:53.049426 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:19:53.049426 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 74/152] 10.222.0.26 () {56 vars in 1114 bytes} [Sat Aug 6 01:19:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:19:53.051061 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65c41a4-ec88-43c5-b1c4-efe6c35ecca7 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:53.051061 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65c41a4-ec88-43c5-b1c4-efe6c35ecca7 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:53.051061 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65c41a4-ec88-43c5-b1c4-efe6c35ecca7 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:53.057321 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 75/153] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:19:53 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:19:53.062838 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65c41a4-ec88-43c5-b1c4-efe6c35ecca7 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:53.126779 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:53.155579 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 79/154] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:19:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:19:53.157020 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836476d8-d9f2-492c-b00c-c76659ea5296 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:53.157350 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836476d8-d9f2-492c-b00c-c76659ea5296 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:53.157630 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836476d8-d9f2-492c-b00c-c76659ea5296 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:53.191933 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-836476d8-d9f2-492c-b00c-c76659ea5296 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:19:53.193479 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-836476d8-d9f2-492c-b00c-c76659ea5296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:19:53.206225 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:53.295195 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65c41a4-ec88-43c5-b1c4-efe6c35ecca7 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:53.362412 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 80/155] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:19:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:19:53.363273 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836476d8-d9f2-492c-b00c-c76659ea5296 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:53.382248 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dc12f9-1a5c-40f7-99c3-2c0c7594599c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:53.383218 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dc12f9-1a5c-40f7-99c3-2c0c7594599c None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:53.383704 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dc12f9-1a5c-40f7-99c3-2c0c7594599c None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:53.384190 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dc12f9-1a5c-40f7-99c3-2c0c7594599c None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:53.385141 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dc12f9-1a5c-40f7-99c3-2c0c7594599c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:19:53.385656 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dc12f9-1a5c-40f7-99c3-2c0c7594599c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:53.387240 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dc12f9-1a5c-40f7-99c3-2c0c7594599c None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:53.420859 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 76/156] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Aug 6 01:19:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:19:53.422584 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dc12f9-1a5c-40f7-99c3-2c0c7594599c None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:53.547078 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836476d8-d9f2-492c-b00c-c76659ea5296 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:53.577139 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:53.603941 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dc12f9-1a5c-40f7-99c3-2c0c7594599c None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:53.637682 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2cf6cd-ce3e-4154-a11c-f9a3454db16b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:53.638623 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2cf6cd-ce3e-4154-a11c-f9a3454db16b None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:53.639097 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2cf6cd-ce3e-4154-a11c-f9a3454db16b None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:53.639386 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2cf6cd-ce3e-4154-a11c-f9a3454db16b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:53.668050 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2cf6cd-ce3e-4154-a11c-f9a3454db16b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:19:53.668250 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f679645-1ce0-4ebe-8636-f0f8f6bac14a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:53.668816 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f679645-1ce0-4ebe-8636-f0f8f6bac14a None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:53.669128 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f679645-1ce0-4ebe-8636-f0f8f6bac14a None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:53.669982 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f679645-1ce0-4ebe-8636-f0f8f6bac14a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:53.673900 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-9b2cf6cd-ce3e-4154-a11c-f9a3454db16b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 06 01:19:53.673900 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:19:53.673900 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:19:53.673900 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:19:53.673900 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:19:53.673900 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:19:53.673900 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:19:53.673900 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:19:53.673900 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:19:53.673900 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:19:53.673900 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:19:53.673900 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:19:53.673900 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:19:53.673900 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:19:53.673900 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:19:53.673900 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:19:53.674917 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:19:53.674917 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:19:53.674917 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:19:53.674917 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:19:53.674917 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:19:53.674917 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:19:53.674917 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:19:53.674917 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:19:53.674917 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:19:53.674917 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:19:53.674917 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:19:53.674917 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:19:53.674917 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:19:53.674917 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:19:53.674917 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:19:53.674917 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:19:53.675757 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:19:53.675757 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:19:53.675757 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:19:53.675757 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:19:53.675757 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:19:53.675757 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:19:53.675757 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:19:53.675757 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:19:53.675757 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:19:53.675757 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:19:53.675757 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 06 01:19:53.675757 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:19:53.675757 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 81/157] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Aug 6 01:19:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:19:53.680457 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2cf6cd-ce3e-4154-a11c-f9a3454db16b None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:53.684322 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:53.702443 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f679645-1ce0-4ebe-8636-f0f8f6bac14a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:19:53.708644 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-6f679645-1ce0-4ebe-8636-f0f8f6bac14a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:19:53.708644 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:19:53.708644 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:19:53.708644 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:19:53.708644 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:19:53.708644 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:19:53.708644 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:19:53.708644 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:19:53.708644 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:19:53.708644 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:19:53.708644 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:19:53.708644 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:19:53.708644 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:19:53.708644 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:19:53.708644 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:19:53.708644 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:19:53.709586 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:19:53.709586 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:19:53.709586 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:19:53.709586 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:19:53.709586 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:19:53.709586 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:19:53.709586 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:19:53.709586 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:19:53.709586 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:19:53.709586 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:19:53.709586 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:19:53.709586 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:19:53.709586 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:19:53.709586 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:19:53.709586 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:19:53.709586 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:19:53.710371 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:19:53.710371 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:19:53.710371 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:19:53.710371 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:19:53.710371 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:19:53.710371 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:19:53.710371 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:19:53.710371 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:19:53.710371 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:19:53.710371 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:19:53.710371 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:19:53.710371 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:19:53.714427 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 77/158] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Aug 6 01:19:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:19:53.715719 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f679645-1ce0-4ebe-8636-f0f8f6bac14a None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:53.895067 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f679645-1ce0-4ebe-8636-f0f8f6bac14a None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:53.919265 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2cf6cd-ce3e-4154-a11c-f9a3454db16b None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:53.959131 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d4f271-2506-4647-9370-89c8361a190c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:53.959843 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d4f271-2506-4647-9370-89c8361a190c None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:53.960571 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d4f271-2506-4647-9370-89c8361a190c None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:53.960856 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d4f271-2506-4647-9370-89c8361a190c None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:53.961775 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d4f271-2506-4647-9370-89c8361a190c None admin] RBAC: Adding query filter params (name=member) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:19:53.962088 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d4f271-2506-4647-9370-89c8361a190c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:53.963645 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d4f271-2506-4647-9370-89c8361a190c None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:53.981034 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 78/159] 10.222.0.26 () {56 vars in 1101 bytes} [Sat Aug 6 01:19:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:19:53.982457 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d4f271-2506-4647-9370-89c8361a190c None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:54.000175 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:54.004871 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf82eeb2-38c3-441b-9f46-b31bc8c7c54b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:54.005598 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf82eeb2-38c3-441b-9f46-b31bc8c7c54b None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:54.006050 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf82eeb2-38c3-441b-9f46-b31bc8c7c54b None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:54.006168 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf82eeb2-38c3-441b-9f46-b31bc8c7c54b None admin] PATH_INFO: `/v3/users/1cf42c5c30ba4b9a93fc4cdb9713c6cc` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:54.010029 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf82eeb2-38c3-441b-9f46-b31bc8c7c54b None admin] RBAC: Authorizing `identity:get_user(user_id=1cf42c5c30ba4b9a93fc4cdb9713c6cc)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:54.011561 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf82eeb2-38c3-441b-9f46-b31bc8c7c54b None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:54.013455 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 82/160] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Aug 6 01:19:53 2022] GET /identity/v3/users/1cf42c5c30ba4b9a93fc4cdb9713c6cc => generated 275 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:19:54.014339 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf82eeb2-38c3-441b-9f46-b31bc8c7c54b None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:54.171574 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d4f271-2506-4647-9370-89c8361a190c None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:54.179114 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:54.240004 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c13c0eb-665c-44ef-accd-4ad7a9e7cc20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:54.240649 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c13c0eb-665c-44ef-accd-4ad7a9e7cc20 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:54.240936 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c13c0eb-665c-44ef-accd-4ad7a9e7cc20 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:54.241239 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c13c0eb-665c-44ef-accd-4ad7a9e7cc20 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:54.241338 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf82eeb2-38c3-441b-9f46-b31bc8c7c54b None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:54.241981 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c13c0eb-665c-44ef-accd-4ad7a9e7cc20 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:19:54.242292 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c13c0eb-665c-44ef-accd-4ad7a9e7cc20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:54.243944 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c13c0eb-665c-44ef-accd-4ad7a9e7cc20 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:54.259612 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 79/161] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Aug 6 01:19:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:19:54.260593 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c13c0eb-665c-44ef-accd-4ad7a9e7cc20 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:54.312485 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aae8b97-7075-44cf-922c-8eb3211b05d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:54.313166 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aae8b97-7075-44cf-922c-8eb3211b05d9 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:54.313467 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aae8b97-7075-44cf-922c-8eb3211b05d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:54.313728 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aae8b97-7075-44cf-922c-8eb3211b05d9 None admin] PATH_INFO: `/v3/users/1cf42c5c30ba4b9a93fc4cdb9713c6cc` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:54.316401 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aae8b97-7075-44cf-922c-8eb3211b05d9 None admin] RBAC: Authorizing `identity:get_user(user_id=1cf42c5c30ba4b9a93fc4cdb9713c6cc)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:54.317900 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aae8b97-7075-44cf-922c-8eb3211b05d9 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:54.319546 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 83/162] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Aug 6 01:19:54 2022] GET /identity/v3/users/1cf42c5c30ba4b9a93fc4cdb9713c6cc => generated 275 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:19:54.322642 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aae8b97-7075-44cf-922c-8eb3211b05d9 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:54.433921 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:54.481383 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c13c0eb-665c-44ef-accd-4ad7a9e7cc20 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:54.535044 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74219563-7072-4ec7-ab38-c90151f3dc44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:54.535693 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74219563-7072-4ec7-ab38-c90151f3dc44 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:54.535978 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74219563-7072-4ec7-ab38-c90151f3dc44 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:54.536255 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74219563-7072-4ec7-ab38-c90151f3dc44 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:54.550297 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aae8b97-7075-44cf-922c-8eb3211b05d9 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:54.595642 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abce379-a141-49f7-9c16-03a4696a9e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:54.596347 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abce379-a141-49f7-9c16-03a4696a9e7b None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:54.596639 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abce379-a141-49f7-9c16-03a4696a9e7b None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:54.596930 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abce379-a141-49f7-9c16-03a4696a9e7b None admin] PATH_INFO: `/v3/users/ca5d3774b93a41d1a8b95796b7899d4c` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:54.598157 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abce379-a141-49f7-9c16-03a4696a9e7b None admin] RBAC: Authorizing `identity:get_user(user_id=ca5d3774b93a41d1a8b95796b7899d4c)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:54.599671 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abce379-a141-49f7-9c16-03a4696a9e7b None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:54.601605 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 84/163] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Aug 6 01:19:54 2022] GET /identity/v3/users/ca5d3774b93a41d1a8b95796b7899d4c => generated 247 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:19:54.603539 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abce379-a141-49f7-9c16-03a4696a9e7b None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:54.635217 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:54.691448 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74219563-7072-4ec7-ab38-c90151f3dc44 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:54.692855 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74219563-7072-4ec7-ab38-c90151f3dc44 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:54.751860 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abce379-a141-49f7-9c16-03a4696a9e7b None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:54.822951 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fed788-2585-4518-9cd9-9b78cacd81f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:54.823593 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fed788-2585-4518-9cd9-9b78cacd81f2 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:54.823924 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fed788-2585-4518-9cd9-9b78cacd81f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:54.824145 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fed788-2585-4518-9cd9-9b78cacd81f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:54.832112 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:54.892301 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 80/164] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Aug 6 01:19:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 632 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 01:19:54.899442 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8858a3a5-be8c-4ea0-9b1b-5faf22738f6e None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:54.899736 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8858a3a5-be8c-4ea0-9b1b-5faf22738f6e None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:54.900017 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8858a3a5-be8c-4ea0-9b1b-5faf22738f6e None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:54.902249 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 81/165] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:19:54 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:19:54.904642 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8858a3a5-be8c-4ea0-9b1b-5faf22738f6e None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:54.948346 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fed788-2585-4518-9cd9-9b78cacd81f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:54.950531 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fed788-2585-4518-9cd9-9b78cacd81f2 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:54.959261 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:55.057366 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 85/166] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Aug 6 01:19:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 01:19:55.058955 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24efef81-2282-4fb9-9db0-ad476eb0fce6 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:55.059481 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24efef81-2282-4fb9-9db0-ad476eb0fce6 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:55.060143 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24efef81-2282-4fb9-9db0-ad476eb0fce6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:55.089822 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-24efef81-2282-4fb9-9db0-ad476eb0fce6 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:19:55.092809 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-24efef81-2282-4fb9-9db0-ad476eb0fce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:19:55.100456 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8858a3a5-be8c-4ea0-9b1b-5faf22738f6e None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:55.175198 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa369906-bcd8-4af4-9461-69498e086eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:55.176193 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa369906-bcd8-4af4-9461-69498e086eff None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:55.176681 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa369906-bcd8-4af4-9461-69498e086eff None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:55.177213 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa369906-bcd8-4af4-9461-69498e086eff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:55.235293 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 86/167] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:19:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:19:55.241073 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24efef81-2282-4fb9-9db0-ad476eb0fce6 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:55.253019 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:55.318988 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:55.326101 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa369906-bcd8-4af4-9461-69498e086eff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:55.327592 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa369906-bcd8-4af4-9461-69498e086eff None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:55.483438 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24efef81-2282-4fb9-9db0-ad476eb0fce6 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:55.558469 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ccf964-8db7-44a0-876e-6e179dfde4c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:55.559144 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ccf964-8db7-44a0-876e-6e179dfde4c1 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:55.559369 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ccf964-8db7-44a0-876e-6e179dfde4c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:55.559595 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ccf964-8db7-44a0-876e-6e179dfde4c1 None admin] PATH_INFO: `/v3/projects/730685c4f62b43c493d42b5897c99a02` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:55.560823 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ccf964-8db7-44a0-876e-6e179dfde4c1 None admin] RBAC: Authorizing `identity:get_project(project_id=730685c4f62b43c493d42b5897c99a02)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:55.562304 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ccf964-8db7-44a0-876e-6e179dfde4c1 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:55.564369 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 87/168] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Aug 6 01:19:55 2022] GET /identity/v3/projects/730685c4f62b43c493d42b5897c99a02 => generated 298 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:19:55.565123 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ccf964-8db7-44a0-876e-6e179dfde4c1 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:55.567202 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 82/169] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Aug 6 01:19:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 664 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 01:19:55.568000 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa369906-bcd8-4af4-9461-69498e086eff None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:55.643612 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:55.677939 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa369906-bcd8-4af4-9461-69498e086eff None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:55.678404 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ccf964-8db7-44a0-876e-6e179dfde4c1 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:55.680736 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:55.717678 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d4ff00-62ca-47a1-82d4-00e331ee9c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:55.718111 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d228b54-ad8b-496d-bdb5-603b4729045e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:55.718330 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d4ff00-62ca-47a1-82d4-00e331ee9c1d None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:55.718615 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d4ff00-62ca-47a1-82d4-00e331ee9c1d None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:55.718739 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d228b54-ad8b-496d-bdb5-603b4729045e None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:55.718905 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d4ff00-62ca-47a1-82d4-00e331ee9c1d None admin] PATH_INFO: `/v3/projects/730685c4f62b43c493d42b5897c99a02` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:55.719006 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d228b54-ad8b-496d-bdb5-603b4729045e None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:55.719285 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d228b54-ad8b-496d-bdb5-603b4729045e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:55.720157 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d4ff00-62ca-47a1-82d4-00e331ee9c1d None admin] RBAC: Authorizing `identity:get_project(project_id=730685c4f62b43c493d42b5897c99a02)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:55.721839 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d4ff00-62ca-47a1-82d4-00e331ee9c1d None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:55.728676 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 88/170] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Aug 6 01:19:55 2022] GET /identity/v3/projects/730685c4f62b43c493d42b5897c99a02 => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:19:55.728676 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e58cc5-d3c9-4e2a-aa7e-cea6a1ba88ad None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:55.728676 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e58cc5-d3c9-4e2a-aa7e-cea6a1ba88ad None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:55.728676 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e58cc5-d3c9-4e2a-aa7e-cea6a1ba88ad None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:55.728676 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 89/171] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:19:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:19:55.728676 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b7e037-aa28-4879-af17-64fc0da855b7 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:55.729120 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b7e037-aa28-4879-af17-64fc0da855b7 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:55.729120 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b7e037-aa28-4879-af17-64fc0da855b7 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:55.731389 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 90/172] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:19:55 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:19:55.734747 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b7e037-aa28-4879-af17-64fc0da855b7 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:55.738985 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d228b54-ad8b-496d-bdb5-603b4729045e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:19:55.744027 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-9d228b54-ad8b-496d-bdb5-603b4729045e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 06 01:19:55.744027 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:19:55.744027 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:19:55.744027 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:19:55.744027 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:19:55.744027 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:19:55.744027 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:19:55.744027 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:19:55.744027 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:19:55.744027 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:19:55.744027 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:19:55.744027 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:19:55.744027 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:19:55.744027 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:19:55.744027 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:19:55.744027 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:19:55.745105 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:19:55.745105 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:19:55.745105 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:19:55.745105 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:19:55.745105 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:19:55.745105 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:19:55.745105 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:19:55.745105 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:19:55.745105 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:19:55.745105 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:19:55.745105 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:19:55.745105 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:19:55.745105 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:19:55.745105 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:19:55.745105 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:19:55.745105 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:19:55.745990 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:19:55.745990 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:19:55.745990 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:19:55.745990 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:19:55.745990 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:19:55.745990 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:19:55.745990 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:19:55.745990 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:19:55.745990 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:19:55.745990 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:19:55.745990 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 06 01:19:55.745990 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:19:55.745990 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 83/173] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Aug 6 01:19:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:19:55.746627 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d228b54-ad8b-496d-bdb5-603b4729045e None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:55.840151 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b7e037-aa28-4879-af17-64fc0da855b7 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:55.847572 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d228b54-ad8b-496d-bdb5-603b4729045e None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:55.874119 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362ab532-f791-44b3-b601-b5e8a6f29445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:55.874929 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362ab532-f791-44b3-b601-b5e8a6f29445 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:55.875213 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362ab532-f791-44b3-b601-b5e8a6f29445 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:55.875494 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362ab532-f791-44b3-b601-b5e8a6f29445 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:55.876279 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362ab532-f791-44b3-b601-b5e8a6f29445 None admin] RBAC: Adding query filter params (role.id=6d8b63d350944d78812edbefb32ae8ee, scope.project.id=730685c4f62b43c493d42b5897c99a02, user.id=1cf42c5c30ba4b9a93fc4cdb9713c6cc) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:19:55.876457 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:55.876599 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362ab532-f791-44b3-b601-b5e8a6f29445 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:55.878108 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362ab532-f791-44b3-b601-b5e8a6f29445 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:55.882978 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bb541e-380e-4a2f-b2ac-50389a4cbeed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:55.883608 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bb541e-380e-4a2f-b2ac-50389a4cbeed None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:55.883829 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 91/174] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Aug 6 01:19:55 2022] GET /identity/v3/role_assignments?user.id=1cf42c5c30ba4b9a93fc4cdb9713c6cc&scope.project.id=730685c4f62b43c493d42b5897c99a02&role.id=6d8b63d350944d78812edbefb32ae8ee => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:19:55.883917 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bb541e-380e-4a2f-b2ac-50389a4cbeed None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:55.884147 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bb541e-380e-4a2f-b2ac-50389a4cbeed None admin] PATH_INFO: `/v3/projects/730685c4f62b43c493d42b5897c99a02` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:55.884977 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732302a0-ecaa-4878-8fb8-9afaf4a7e12a None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:55.885296 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732302a0-ecaa-4878-8fb8-9afaf4a7e12a None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:55.885402 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bb541e-380e-4a2f-b2ac-50389a4cbeed None admin] RBAC: Authorizing `identity:get_project(project_id=730685c4f62b43c493d42b5897c99a02)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:55.885583 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732302a0-ecaa-4878-8fb8-9afaf4a7e12a None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:55.891289 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bb541e-380e-4a2f-b2ac-50389a4cbeed None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:55.891505 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 92/175] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:19:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:19:55.891505 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732302a0-ecaa-4878-8fb8-9afaf4a7e12a None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:55.891651 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 84/176] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Aug 6 01:19:55 2022] GET /identity/v3/projects/730685c4f62b43c493d42b5897c99a02 => generated 298 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:19:55.891651 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d47386-f853-4c7b-b123-6bdc7a62716e None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:55.891651 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d47386-f853-4c7b-b123-6bdc7a62716e None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:55.891651 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d47386-f853-4c7b-b123-6bdc7a62716e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:55.910042 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-b4d47386-f853-4c7b-b123-6bdc7a62716e None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:19:55.911290 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-b4d47386-f853-4c7b-b123-6bdc7a62716e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:19:55.942683 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:55.994908 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 85/177] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:19:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:19:55.995911 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3edb523-73c3-49f3-8ded-2a5a8158a102 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:55.996198 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3edb523-73c3-49f3-8ded-2a5a8158a102 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:55.996590 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3edb523-73c3-49f3-8ded-2a5a8158a102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:56.017655 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-b3edb523-73c3-49f3-8ded-2a5a8158a102 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:19:56.018636 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-b3edb523-73c3-49f3-8ded-2a5a8158a102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:19:56.038390 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732302a0-ecaa-4878-8fb8-9afaf4a7e12a None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:56.081799 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6b6d9e-e110-421b-99ef-d1c6a6bc3d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:56.082435 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6b6d9e-e110-421b-99ef-d1c6a6bc3d63 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:56.082723 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6b6d9e-e110-421b-99ef-d1c6a6bc3d63 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:56.083006 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6b6d9e-e110-421b-99ef-d1c6a6bc3d63 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:56.083810 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6b6d9e-e110-421b-99ef-d1c6a6bc3d63 None admin] RBAC: Adding query filter params (role.id=61b3b93d37a047e4bb8369b46fe20db9, scope.project.id=730685c4f62b43c493d42b5897c99a02, user.id=1cf42c5c30ba4b9a93fc4cdb9713c6cc) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:19:56.084128 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6b6d9e-e110-421b-99ef-d1c6a6bc3d63 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:56.085659 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6b6d9e-e110-421b-99ef-d1c6a6bc3d63 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:56.093181 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 93/178] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Aug 6 01:19:55 2022] GET /identity/v3/role_assignments?user.id=1cf42c5c30ba4b9a93fc4cdb9713c6cc&scope.project.id=730685c4f62b43c493d42b5897c99a02&role.id=61b3b93d37a047e4bb8369b46fe20db9 => generated 608 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:19:56.093556 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6b6d9e-e110-421b-99ef-d1c6a6bc3d63 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:56.097921 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 86/179] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:19:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:19:56.104684 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99c504b-68d3-4f57-9b12-aa4450252671 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:56.105215 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99c504b-68d3-4f57-9b12-aa4450252671 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:56.105626 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99c504b-68d3-4f57-9b12-aa4450252671 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:56.108268 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 87/180] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:19:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:19:56.113916 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6caf34-36da-43f9-b60e-8d436313c3f4 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:56.114427 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6caf34-36da-43f9-b60e-8d436313c3f4 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:56.114844 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6caf34-36da-43f9-b60e-8d436313c3f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:56.141184 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-5f6caf34-36da-43f9-b60e-8d436313c3f4 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:19:56.144096 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-5f6caf34-36da-43f9-b60e-8d436313c3f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:19:56.210755 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:56.221228 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:56.224246 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6b6d9e-e110-421b-99ef-d1c6a6bc3d63 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:56.228297 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 88/181] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:19:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:19:56.228960 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6caf34-36da-43f9-b60e-8d436313c3f4 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:56.273408 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba02def-9f7f-42e0-bcbd-5e6adff8549b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:56.274050 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba02def-9f7f-42e0-bcbd-5e6adff8549b None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:56.274331 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba02def-9f7f-42e0-bcbd-5e6adff8549b None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:56.274610 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba02def-9f7f-42e0-bcbd-5e6adff8549b None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:56.275340 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba02def-9f7f-42e0-bcbd-5e6adff8549b None admin] RBAC: Adding query filter params (name=member) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:19:56.275647 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba02def-9f7f-42e0-bcbd-5e6adff8549b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:56.277023 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba02def-9f7f-42e0-bcbd-5e6adff8549b None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:56.286889 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 94/182] 10.222.0.26 () {56 vars in 1101 bytes} [Sat Aug 6 01:19:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:19:56.287840 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba02def-9f7f-42e0-bcbd-5e6adff8549b None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:56.347776 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6caf34-36da-43f9-b60e-8d436313c3f4 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:56.383423 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406d8615-e2c3-468e-97ee-c48b646848b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:56.384047 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406d8615-e2c3-468e-97ee-c48b646848b2 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:56.384329 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406d8615-e2c3-468e-97ee-c48b646848b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:56.384602 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406d8615-e2c3-468e-97ee-c48b646848b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:56.385414 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406d8615-e2c3-468e-97ee-c48b646848b2 None admin] RBAC: Adding query filter params (role.id=4559b9a0837249f895a13c70096f8d38, scope.project.id=730685c4f62b43c493d42b5897c99a02, user.id=ca5d3774b93a41d1a8b95796b7899d4c) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:19:56.385725 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406d8615-e2c3-468e-97ee-c48b646848b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:56.387159 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406d8615-e2c3-468e-97ee-c48b646848b2 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:56.393125 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 89/183] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Aug 6 01:19:56 2022] GET /identity/v3/role_assignments?user.id=ca5d3774b93a41d1a8b95796b7899d4c&scope.project.id=730685c4f62b43c493d42b5897c99a02&role.id=4559b9a0837249f895a13c70096f8d38 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:19:56.395561 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba02def-9f7f-42e0-bcbd-5e6adff8549b None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:56.402828 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95346e37-b09c-45bf-8d5d-804af32106ec None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:56.404822 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95346e37-b09c-45bf-8d5d-804af32106ec None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:56.405337 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95346e37-b09c-45bf-8d5d-804af32106ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:56.428835 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-95346e37-b09c-45bf-8d5d-804af32106ec None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:19:56.430090 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-95346e37-b09c-45bf-8d5d-804af32106ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:19:56.438177 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b9919f-47a7-4878-85c3-6488adb9932f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:56.439271 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b9919f-47a7-4878-85c3-6488adb9932f None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:56.439271 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b9919f-47a7-4878-85c3-6488adb9932f None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:56.439457 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b9919f-47a7-4878-85c3-6488adb9932f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:56.453294 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b9919f-47a7-4878-85c3-6488adb9932f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:19:56.459448 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-21b9919f-47a7-4878-85c3-6488adb9932f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:19:56.459448 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:19:56.459448 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:19:56.459448 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:19:56.459448 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:19:56.459448 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:19:56.459448 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:19:56.459448 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:19:56.459448 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:19:56.459448 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:19:56.459448 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:19:56.459448 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:19:56.459448 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:19:56.459448 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:19:56.459448 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:19:56.459448 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:19:56.460408 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:19:56.460408 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:19:56.460408 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:19:56.460408 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:19:56.460408 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:19:56.460408 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:19:56.460408 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:19:56.460408 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:19:56.460408 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:19:56.460408 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:19:56.460408 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:19:56.460408 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:19:56.460408 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:19:56.460408 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:19:56.460408 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:19:56.460408 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:19:56.461253 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:19:56.461253 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:19:56.461253 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:19:56.461253 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:19:56.461253 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:19:56.461253 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:19:56.461253 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:19:56.461253 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:19:56.461253 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:19:56.461253 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:19:56.461253 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:19:56.461253 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:19:56.461253 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 95/184] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Aug 6 01:19:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:19:56.461880 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b9919f-47a7-4878-85c3-6488adb9932f None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:56.488211 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:56.490513 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:56.512166 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 90/185] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:19:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:19:56.522257 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95346e37-b09c-45bf-8d5d-804af32106ec None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:56.583746 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b9919f-47a7-4878-85c3-6488adb9932f None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:56.621870 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61fe22c-42eb-42d2-826c-b8827ad2c7a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:56.623131 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61fe22c-42eb-42d2-826c-b8827ad2c7a7 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:56.623720 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61fe22c-42eb-42d2-826c-b8827ad2c7a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:56.624206 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61fe22c-42eb-42d2-826c-b8827ad2c7a7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:56.634971 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61fe22c-42eb-42d2-826c-b8827ad2c7a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:19:56.640511 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-b61fe22c-42eb-42d2-826c-b8827ad2c7a7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 01:19:56.640511 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:19:56.640511 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:19:56.640511 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:19:56.640511 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:19:56.640511 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:19:56.640511 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:19:56.640511 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:19:56.640511 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:19:56.640511 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:19:56.640511 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:19:56.640511 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:19:56.640511 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:19:56.640511 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:19:56.640511 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:19:56.640511 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:19:56.641613 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:19:56.641613 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:19:56.641613 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:19:56.641613 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:19:56.641613 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:19:56.641613 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:19:56.641613 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:19:56.641613 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:19:56.641613 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:19:56.641613 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:19:56.641613 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:19:56.641613 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:19:56.641613 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:19:56.641613 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:19:56.641613 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:19:56.641613 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:19:56.642444 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:19:56.642444 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:19:56.642444 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:19:56.642444 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:19:56.642444 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:19:56.642444 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:19:56.642444 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:19:56.642444 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:19:56.642444 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:19:56.642444 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:19:56.642444 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 01:19:56.642444 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:19:56.642444 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 96/186] 10.222.0.26 () {56 vars in 1114 bytes} [Sat Aug 6 01:19:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:19:56.643090 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61fe22c-42eb-42d2-826c-b8827ad2c7a7 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:56.645248 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95346e37-b09c-45bf-8d5d-804af32106ec None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:56.682097 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22ede7e-1aa1-47a1-bf3a-ea32199052cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:56.682920 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22ede7e-1aa1-47a1-bf3a-ea32199052cd None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:56.683349 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22ede7e-1aa1-47a1-bf3a-ea32199052cd None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:56.683683 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22ede7e-1aa1-47a1-bf3a-ea32199052cd None admin] PATH_INFO: `/v3/domains/default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:56.685082 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22ede7e-1aa1-47a1-bf3a-ea32199052cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:56.686857 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22ede7e-1aa1-47a1-bf3a-ea32199052cd None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:56.688462 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 91/187] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:19:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:19:56.689245 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22ede7e-1aa1-47a1-bf3a-ea32199052cd None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:56.726840 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:56.740004 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:56.745850 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61fe22c-42eb-42d2-826c-b8827ad2c7a7 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:56.779308 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd1208e-24a7-4e32-a613-8afb000ff71b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:56.780029 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd1208e-24a7-4e32-a613-8afb000ff71b None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:56.780312 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd1208e-24a7-4e32-a613-8afb000ff71b None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:56.780590 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd1208e-24a7-4e32-a613-8afb000ff71b None admin] PATH_INFO: `/v3/users/1cf42c5c30ba4b9a93fc4cdb9713c6cc` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:56.781804 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd1208e-24a7-4e32-a613-8afb000ff71b None admin] RBAC: Authorizing `identity:get_user(user_id=1cf42c5c30ba4b9a93fc4cdb9713c6cc)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:56.783253 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd1208e-24a7-4e32-a613-8afb000ff71b None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:56.784873 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 97/188] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Aug 6 01:19:56 2022] GET /identity/v3/users/1cf42c5c30ba4b9a93fc4cdb9713c6cc => generated 275 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:19:56.786712 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd1208e-24a7-4e32-a613-8afb000ff71b None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:56.788298 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22ede7e-1aa1-47a1-bf3a-ea32199052cd None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:56.822361 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b11ee6-4f95-44cd-b5e5-d55a821c0ac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:56.823004 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b11ee6-4f95-44cd-b5e5-d55a821c0ac4 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:56.823290 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b11ee6-4f95-44cd-b5e5-d55a821c0ac4 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:56.823565 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b11ee6-4f95-44cd-b5e5-d55a821c0ac4 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:56.824285 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b11ee6-4f95-44cd-b5e5-d55a821c0ac4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:19:56.824589 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b11ee6-4f95-44cd-b5e5-d55a821c0ac4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:56.826127 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b11ee6-4f95-44cd-b5e5-d55a821c0ac4 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:56.834344 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 92/189] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Aug 6 01:19:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:19:56.835049 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b11ee6-4f95-44cd-b5e5-d55a821c0ac4 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:56.887755 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd1208e-24a7-4e32-a613-8afb000ff71b None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:56.920765 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5eb8aa-f553-4911-aa92-aaec902a3f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:56.921420 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5eb8aa-f553-4911-aa92-aaec902a3f3e None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:56.921724 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5eb8aa-f553-4911-aa92-aaec902a3f3e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:56.921998 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5eb8aa-f553-4911-aa92-aaec902a3f3e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:56.931999 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5eb8aa-f553-4911-aa92-aaec902a3f3e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:19:56.934628 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b11ee6-4f95-44cd-b5e5-d55a821c0ac4 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:56.937402 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-ee5eb8aa-f553-4911-aa92-aaec902a3f3e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 06 01:19:56.937402 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:19:56.937402 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:19:56.937402 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:19:56.937402 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:19:56.937402 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:19:56.937402 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:19:56.937402 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:19:56.937402 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:19:56.937402 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:19:56.937402 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:19:56.937402 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:19:56.937402 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:19:56.937402 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:19:56.937402 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:19:56.937402 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:19:56.938375 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:19:56.938375 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:19:56.938375 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:19:56.938375 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:19:56.938375 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:19:56.938375 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:19:56.938375 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:19:56.938375 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:19:56.938375 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:19:56.938375 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:19:56.938375 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:19:56.938375 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:19:56.938375 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:19:56.938375 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:19:56.938375 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:19:56.938375 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:19:56.939163 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:19:56.939163 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:19:56.939163 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:19:56.939163 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:19:56.939163 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:19:56.939163 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:19:56.939163 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:19:56.939163 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:19:56.939163 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:19:56.939163 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:19:56.939163 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 06 01:19:56.939163 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:19:56.939163 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 98/190] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Aug 6 01:19:56 2022] GET /identity/v3/roles/reader => generated 84 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:19:56.942224 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5eb8aa-f553-4911-aa92-aaec902a3f3e None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:56.947797 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:56.955009 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:56.970484 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340e90e0-2907-48ca-9754-84685fd049db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:56.971103 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340e90e0-2907-48ca-9754-84685fd049db None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:56.971385 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340e90e0-2907-48ca-9754-84685fd049db None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:56.971656 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340e90e0-2907-48ca-9754-84685fd049db None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:56.972368 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340e90e0-2907-48ca-9754-84685fd049db None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:19:56.972672 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340e90e0-2907-48ca-9754-84685fd049db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:56.974065 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340e90e0-2907-48ca-9754-84685fd049db None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:56.982018 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 93/191] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Aug 6 01:19:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:19:56.983990 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340e90e0-2907-48ca-9754-84685fd049db None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:57.044011 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5eb8aa-f553-4911-aa92-aaec902a3f3e None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:57.077490 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc411e30-88f8-4299-a854-fbcda44950ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:57.078057 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc411e30-88f8-4299-a854-fbcda44950ca None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:57.078346 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc411e30-88f8-4299-a854-fbcda44950ca None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:57.078621 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc411e30-88f8-4299-a854-fbcda44950ca None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:57.079312 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc411e30-88f8-4299-a854-fbcda44950ca None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:57.080752 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc411e30-88f8-4299-a854-fbcda44950ca None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:57.081034 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340e90e0-2907-48ca-9754-84685fd049db None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:57.102586 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 99/192] 10.222.0.26 () {60 vars in 1132 bytes} [Sat Aug 6 01:19:56 2022] POST /identity/v3/users => generated 297 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:19:57.103671 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc411e30-88f8-4299-a854-fbcda44950ca None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:57.128081 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e4553a-9e01-4884-9c96-9e4a69f1b27e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:57.128727 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e4553a-9e01-4884-9c96-9e4a69f1b27e None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:57.129011 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e4553a-9e01-4884-9c96-9e4a69f1b27e None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:57.129321 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e4553a-9e01-4884-9c96-9e4a69f1b27e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:57.175278 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:57.183295 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:57.217085 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc411e30-88f8-4299-a854-fbcda44950ca None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:57.217454 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e4553a-9e01-4884-9c96-9e4a69f1b27e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:57.218953 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e4553a-9e01-4884-9c96-9e4a69f1b27e None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:57.255441 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edcaaad-e443-4802-a7b9-33e83c6572b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:57.256102 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edcaaad-e443-4802-a7b9-33e83c6572b8 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:57.256422 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edcaaad-e443-4802-a7b9-33e83c6572b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:57.256765 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edcaaad-e443-4802-a7b9-33e83c6572b8 None admin] PATH_INFO: `/v3/users/c1fcb82f07be4ffc85ac133727f3a453` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:57.277876 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edcaaad-e443-4802-a7b9-33e83c6572b8 None admin] RBAC: Authorizing `identity:get_user(user_id=c1fcb82f07be4ffc85ac133727f3a453)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:57.279392 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edcaaad-e443-4802-a7b9-33e83c6572b8 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:57.281143 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 100/193] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Aug 6 01:19:57 2022] GET /identity/v3/users/c1fcb82f07be4ffc85ac133727f3a453 => generated 283 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:19:57.281922 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edcaaad-e443-4802-a7b9-33e83c6572b8 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:57.330323 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 94/194] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Aug 6 01:19:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 01:19:57.331464 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e4553a-9e01-4884-9c96-9e4a69f1b27e None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:57.420133 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edcaaad-e443-4802-a7b9-33e83c6572b8 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:57.433021 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:57.447969 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:57.463388 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dd4ccb-d768-48f4-a440-40b098b6ac38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:57.464018 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dd4ccb-d768-48f4-a440-40b098b6ac38 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:57.464297 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dd4ccb-d768-48f4-a440-40b098b6ac38 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:57.464572 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dd4ccb-d768-48f4-a440-40b098b6ac38 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:57.465322 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dd4ccb-d768-48f4-a440-40b098b6ac38 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:19:57.465628 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dd4ccb-d768-48f4-a440-40b098b6ac38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:57.467938 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dd4ccb-d768-48f4-a440-40b098b6ac38 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:57.471020 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e4553a-9e01-4884-9c96-9e4a69f1b27e None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:57.476805 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 101/195] 10.222.0.26 () {56 vars in 1101 bytes} [Sat Aug 6 01:19:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:19:57.477677 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dd4ccb-d768-48f4-a440-40b098b6ac38 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:57.517306 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5159ed65-5703-4102-8035-b7fb2a44459b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:57.517967 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5159ed65-5703-4102-8035-b7fb2a44459b None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:57.518258 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5159ed65-5703-4102-8035-b7fb2a44459b None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:57.518547 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5159ed65-5703-4102-8035-b7fb2a44459b None admin] PATH_INFO: `/v3/users/c1fcb82f07be4ffc85ac133727f3a453` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:57.519752 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5159ed65-5703-4102-8035-b7fb2a44459b None admin] RBAC: Authorizing `identity:get_user(user_id=c1fcb82f07be4ffc85ac133727f3a453)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:57.521871 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5159ed65-5703-4102-8035-b7fb2a44459b None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:57.529301 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 95/196] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Aug 6 01:19:57 2022] GET /identity/v3/users/c1fcb82f07be4ffc85ac133727f3a453 => generated 283 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:19:57.530148 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5159ed65-5703-4102-8035-b7fb2a44459b None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:57.585225 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dd4ccb-d768-48f4-a440-40b098b6ac38 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:57.621411 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71422407-2b41-4392-a202-e87303dbccba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:57.622145 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71422407-2b41-4392-a202-e87303dbccba None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:57.623237 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71422407-2b41-4392-a202-e87303dbccba None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:57.623813 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71422407-2b41-4392-a202-e87303dbccba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:57.645491 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5159ed65-5703-4102-8035-b7fb2a44459b None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:57.680162 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:57.684333 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268711ab-5298-45d3-9f09-52a3112a16fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:57.685613 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268711ab-5298-45d3-9f09-52a3112a16fc None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:57.685894 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268711ab-5298-45d3-9f09-52a3112a16fc None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:57.686171 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268711ab-5298-45d3-9f09-52a3112a16fc None admin] PATH_INFO: `/v3/projects/3d00a20dc2ed44be99579eaac1c6dfa1` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:57.688034 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268711ab-5298-45d3-9f09-52a3112a16fc None admin] RBAC: Authorizing `identity:get_project(project_id=3d00a20dc2ed44be99579eaac1c6dfa1)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:57.689907 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268711ab-5298-45d3-9f09-52a3112a16fc None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:57.691829 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 96/197] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Aug 6 01:19:57 2022] GET /identity/v3/projects/3d00a20dc2ed44be99579eaac1c6dfa1 => generated 312 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:19:57.695946 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71422407-2b41-4392-a202-e87303dbccba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:57.697299 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268711ab-5298-45d3-9f09-52a3112a16fc None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:57.697876 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71422407-2b41-4392-a202-e87303dbccba None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:57.711010 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:57.802015 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268711ab-5298-45d3-9f09-52a3112a16fc None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:57.802690 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 102/198] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Aug 6 01:19:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 01:19:57.803546 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71422407-2b41-4392-a202-e87303dbccba None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:57.843104 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d8ac75-a3f6-4d7f-a430-73de4567c84a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:57.843753 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d8ac75-a3f6-4d7f-a430-73de4567c84a None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:57.844035 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d8ac75-a3f6-4d7f-a430-73de4567c84a None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:57.844316 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d8ac75-a3f6-4d7f-a430-73de4567c84a None admin] PATH_INFO: `/v3/users/3b686ca118134310a61f9e8157e9b9be` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:57.845719 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d8ac75-a3f6-4d7f-a430-73de4567c84a None admin] RBAC: Authorizing `identity:get_user(user_id=3b686ca118134310a61f9e8157e9b9be)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:57.847172 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d8ac75-a3f6-4d7f-a430-73de4567c84a None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:57.848777 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 97/199] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Aug 6 01:19:57 2022] GET /identity/v3/users/3b686ca118134310a61f9e8157e9b9be => generated 289 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:19:57.849639 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d8ac75-a3f6-4d7f-a430-73de4567c84a None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:57.895840 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:57.909284 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71422407-2b41-4392-a202-e87303dbccba None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:57.944799 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:57.951094 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076f759a-c25c-45a6-8e0f-c6f331a6bd5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:57.951961 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076f759a-c25c-45a6-8e0f-c6f331a6bd5f None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:57.952458 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076f759a-c25c-45a6-8e0f-c6f331a6bd5f None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:57.952844 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076f759a-c25c-45a6-8e0f-c6f331a6bd5f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:57.968962 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d8ac75-a3f6-4d7f-a430-73de4567c84a None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:58.012126 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f986972-1d42-4dfd-8436-ddbfc75ef5f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:58.012971 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f986972-1d42-4dfd-8436-ddbfc75ef5f9 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:58.013669 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f986972-1d42-4dfd-8436-ddbfc75ef5f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:58.014015 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f986972-1d42-4dfd-8436-ddbfc75ef5f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:58.014977 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f986972-1d42-4dfd-8436-ddbfc75ef5f9 None admin] RBAC: Adding query filter params (role.id=61b3b93d37a047e4bb8369b46fe20db9, scope.project.id=3d00a20dc2ed44be99579eaac1c6dfa1, user.id=1cf42c5c30ba4b9a93fc4cdb9713c6cc) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:19:58.015338 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f986972-1d42-4dfd-8436-ddbfc75ef5f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:58.017133 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f986972-1d42-4dfd-8436-ddbfc75ef5f9 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:58.024829 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 98/200] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Aug 6 01:19:57 2022] GET /identity/v3/role_assignments?user.id=1cf42c5c30ba4b9a93fc4cdb9713c6cc&scope.project.id=3d00a20dc2ed44be99579eaac1c6dfa1&role.id=61b3b93d37a047e4bb8369b46fe20db9 => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:19:58.025825 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f986972-1d42-4dfd-8436-ddbfc75ef5f9 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:58.053865 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076f759a-c25c-45a6-8e0f-c6f331a6bd5f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:58.056898 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076f759a-c25c-45a6-8e0f-c6f331a6bd5f None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:58.220092 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f986972-1d42-4dfd-8436-ddbfc75ef5f9 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:58.233208 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:58.256529 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 103/201] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Aug 6 01:19:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 01:19:58.258656 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076f759a-c25c-45a6-8e0f-c6f331a6bd5f None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:58.269397 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83793273-bf0f-4c80-81e6-ce27e70dda2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:58.270047 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83793273-bf0f-4c80-81e6-ce27e70dda2b None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:58.270325 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83793273-bf0f-4c80-81e6-ce27e70dda2b None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:58.270599 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83793273-bf0f-4c80-81e6-ce27e70dda2b None admin] PATH_INFO: `/v3/projects/0bcfa6bfcd464fc9b0dc2079b24902bc` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:58.272299 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83793273-bf0f-4c80-81e6-ce27e70dda2b None admin] RBAC: Authorizing `identity:get_project(project_id=0bcfa6bfcd464fc9b0dc2079b24902bc)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:58.273808 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83793273-bf0f-4c80-81e6-ce27e70dda2b None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:58.276448 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 99/202] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Aug 6 01:19:58 2022] GET /identity/v3/projects/0bcfa6bfcd464fc9b0dc2079b24902bc => generated 302 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:19:58.277245 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83793273-bf0f-4c80-81e6-ce27e70dda2b None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:58.298492 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:58.428913 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076f759a-c25c-45a6-8e0f-c6f331a6bd5f None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:58.430262 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83793273-bf0f-4c80-81e6-ce27e70dda2b None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:58.472573 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523f4651-452c-450c-9f47-c04e99c3f41b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:58.473314 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523f4651-452c-450c-9f47-c04e99c3f41b None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:58.473713 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523f4651-452c-450c-9f47-c04e99c3f41b None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:58.474094 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523f4651-452c-450c-9f47-c04e99c3f41b None admin] PATH_INFO: `/v3/projects/0bcfa6bfcd464fc9b0dc2079b24902bc` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:58.474967 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50de9494-d809-46f4-8e36-8d1b1ba5e950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:58.475514 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523f4651-452c-450c-9f47-c04e99c3f41b None admin] RBAC: Authorizing `identity:get_project(project_id=0bcfa6bfcd464fc9b0dc2079b24902bc)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:58.476179 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50de9494-d809-46f4-8e36-8d1b1ba5e950 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:58.476603 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50de9494-d809-46f4-8e36-8d1b1ba5e950 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:58.477029 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50de9494-d809-46f4-8e36-8d1b1ba5e950 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:58.477671 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523f4651-452c-450c-9f47-c04e99c3f41b None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:58.479700 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 100/203] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Aug 6 01:19:58 2022] GET /identity/v3/projects/0bcfa6bfcd464fc9b0dc2079b24902bc => generated 302 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:19:58.484126 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523f4651-452c-450c-9f47-c04e99c3f41b None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:58.539036 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:58.554802 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50de9494-d809-46f4-8e36-8d1b1ba5e950 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:58.556577 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50de9494-d809-46f4-8e36-8d1b1ba5e950 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:58.571676 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:58.596599 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523f4651-452c-450c-9f47-c04e99c3f41b None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:58.631729 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412cc6df-c004-4082-9a18-83fcb96f5f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:58.632374 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412cc6df-c004-4082-9a18-83fcb96f5f74 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:58.632654 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412cc6df-c004-4082-9a18-83fcb96f5f74 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:58.632926 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412cc6df-c004-4082-9a18-83fcb96f5f74 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:58.633744 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412cc6df-c004-4082-9a18-83fcb96f5f74 None admin] RBAC: Adding query filter params (role.id=4559b9a0837249f895a13c70096f8d38, scope.project.id=0bcfa6bfcd464fc9b0dc2079b24902bc, user.id=c1fcb82f07be4ffc85ac133727f3a453) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:19:58.634055 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412cc6df-c004-4082-9a18-83fcb96f5f74 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:58.635505 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412cc6df-c004-4082-9a18-83fcb96f5f74 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:58.641718 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 101/204] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Aug 6 01:19:58 2022] GET /identity/v3/role_assignments?user.id=c1fcb82f07be4ffc85ac133727f3a453&scope.project.id=0bcfa6bfcd464fc9b0dc2079b24902bc&role.id=4559b9a0837249f895a13c70096f8d38 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:19:58.649449 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412cc6df-c004-4082-9a18-83fcb96f5f74 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:58.698241 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 104/205] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Aug 6 01:19:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 01:19:58.704501 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50de9494-d809-46f4-8e36-8d1b1ba5e950 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:58.814099 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412cc6df-c004-4082-9a18-83fcb96f5f74 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:58.864714 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:58.882287 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50de9494-d809-46f4-8e36-8d1b1ba5e950 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:58.893888 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8609a12-a91a-411a-8301-b8aad96ab4c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:58.894535 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8609a12-a91a-411a-8301-b8aad96ab4c5 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:58.894822 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8609a12-a91a-411a-8301-b8aad96ab4c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:58.895103 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8609a12-a91a-411a-8301-b8aad96ab4c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:58.895907 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8609a12-a91a-411a-8301-b8aad96ab4c5 None admin] RBAC: Adding query filter params (role.id=6d8b63d350944d78812edbefb32ae8ee, scope.project.id=0bcfa6bfcd464fc9b0dc2079b24902bc, user.id=c1fcb82f07be4ffc85ac133727f3a453) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:19:58.896220 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8609a12-a91a-411a-8301-b8aad96ab4c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:58.897712 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8609a12-a91a-411a-8301-b8aad96ab4c5 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:58.915066 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 102/206] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Aug 6 01:19:58 2022] GET /identity/v3/role_assignments?user.id=c1fcb82f07be4ffc85ac133727f3a453&scope.project.id=0bcfa6bfcd464fc9b0dc2079b24902bc&role.id=6d8b63d350944d78812edbefb32ae8ee => generated 259 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:19:58.923154 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4708bd6a-eb97-4b0a-b115-dbea591fc604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:58.924185 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4708bd6a-eb97-4b0a-b115-dbea591fc604 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:58.924693 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4708bd6a-eb97-4b0a-b115-dbea591fc604 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:58.925210 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4708bd6a-eb97-4b0a-b115-dbea591fc604 None admin] PATH_INFO: `/v3/projects/730685c4f62b43c493d42b5897c99a02` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:58.926697 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4708bd6a-eb97-4b0a-b115-dbea591fc604 None admin] RBAC: Authorizing `identity:get_project(project_id=730685c4f62b43c493d42b5897c99a02)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:58.928339 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4708bd6a-eb97-4b0a-b115-dbea591fc604 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:58.930670 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 105/207] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Aug 6 01:19:58 2022] GET /identity/v3/projects/730685c4f62b43c493d42b5897c99a02 => generated 298 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:19:58.936447 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8609a12-a91a-411a-8301-b8aad96ab4c5 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:58.956779 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:59.121143 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8609a12-a91a-411a-8301-b8aad96ab4c5 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:59.166518 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed159d1c-5072-475a-83fd-1b620f31b0b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:59.167148 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed159d1c-5072-475a-83fd-1b620f31b0b5 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:59.167431 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed159d1c-5072-475a-83fd-1b620f31b0b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:59.167711 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed159d1c-5072-475a-83fd-1b620f31b0b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:59.168506 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed159d1c-5072-475a-83fd-1b620f31b0b5 None admin] RBAC: Adding query filter params (role.id=25efb2aed94e4d63b0b6cb8faf5a2ee7, scope.project.id=730685c4f62b43c493d42b5897c99a02, user.id=3b686ca118134310a61f9e8157e9b9be) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:19:59.168818 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed159d1c-5072-475a-83fd-1b620f31b0b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:59.170744 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed159d1c-5072-475a-83fd-1b620f31b0b5 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:59.177228 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 103/208] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Aug 6 01:19:58 2022] GET /identity/v3/role_assignments?user.id=3b686ca118134310a61f9e8157e9b9be&scope.project.id=730685c4f62b43c493d42b5897c99a02&role.id=25efb2aed94e4d63b0b6cb8faf5a2ee7 => generated 608 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:19:59.456110 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddd31d7-5f8d-4b49-9c87-cf76e84f7341 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:59.456611 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddd31d7-5f8d-4b49-9c87-cf76e84f7341 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:59.456684 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddd31d7-5f8d-4b49-9c87-cf76e84f7341 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:59.458233 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 106/209] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:19:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:19:59.463740 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eee4b08-9cfb-4c6d-b6f0-029e92ab1299 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:59.464028 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eee4b08-9cfb-4c6d-b6f0-029e92ab1299 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:59.464312 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eee4b08-9cfb-4c6d-b6f0-029e92ab1299 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:59.483915 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-5eee4b08-9cfb-4c6d-b6f0-029e92ab1299 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:19:59.485161 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-5eee4b08-9cfb-4c6d-b6f0-029e92ab1299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:19:59.490475 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840e990c-c489-4b24-afae-e8dc3531003f None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:59.490760 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840e990c-c489-4b24-afae-e8dc3531003f None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:59.491039 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840e990c-c489-4b24-afae-e8dc3531003f None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:59.492406 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 107/210] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:19:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:19:59.497768 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64242532-bf1f-41d0-9ffb-1673be4f040c None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:59.498056 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64242532-bf1f-41d0-9ffb-1673be4f040c None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:59.498348 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64242532-bf1f-41d0-9ffb-1673be4f040c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:59.517372 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-64242532-bf1f-41d0-9ffb-1673be4f040c None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:19:59.518578 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-64242532-bf1f-41d0-9ffb-1673be4f040c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:19:59.575409 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:59.582614 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 104/211] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:19:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:19:59.603869 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 108/212] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:19:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 01:19:59.609512 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eee4b08-9cfb-4c6d-b6f0-029e92ab1299 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:59.629285 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64242532-bf1f-41d0-9ffb-1673be4f040c None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:59.723617 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:59.736595 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eee4b08-9cfb-4c6d-b6f0-029e92ab1299 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:59.741156 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64242532-bf1f-41d0-9ffb-1673be4f040c None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:59.781780 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36868ed7-93c3-4867-8ed0-db74fe7ef582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:59.782242 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27545169-36d9-4f77-b280-bf610603b385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:59.782482 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36868ed7-93c3-4867-8ed0-db74fe7ef582 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:59.782771 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36868ed7-93c3-4867-8ed0-db74fe7ef582 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:59.782935 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27545169-36d9-4f77-b280-bf610603b385 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:59.783081 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36868ed7-93c3-4867-8ed0-db74fe7ef582 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:59.783224 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27545169-36d9-4f77-b280-bf610603b385 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:59.783416 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27545169-36d9-4f77-b280-bf610603b385 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:59.784175 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36868ed7-93c3-4867-8ed0-db74fe7ef582 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:59.786287 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36868ed7-93c3-4867-8ed0-db74fe7ef582 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:59.787879 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 105/213] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:19:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:19:59.793508 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36868ed7-93c3-4867-8ed0-db74fe7ef582 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:59.795303 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27545169-36d9-4f77-b280-bf610603b385 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:19:59.800539 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-27545169-36d9-4f77-b280-bf610603b385 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 06 01:19:59.800539 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:19:59.800539 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:19:59.800539 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:19:59.800539 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:19:59.800539 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:19:59.800539 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:19:59.800539 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:19:59.800539 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:19:59.800539 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:19:59.800539 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:19:59.800539 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:19:59.800539 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:19:59.800539 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:19:59.800539 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:19:59.800539 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:19:59.802776 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:19:59.802776 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:19:59.802776 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:19:59.802776 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:19:59.802776 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:19:59.802776 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:19:59.802776 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:19:59.802776 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:19:59.802776 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:19:59.802776 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:19:59.802776 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:19:59.802776 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:19:59.802776 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:19:59.802776 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:19:59.802776 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:19:59.802776 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:19:59.804745 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:19:59.804745 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:19:59.804745 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:19:59.804745 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:19:59.804745 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:19:59.804745 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:19:59.804745 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:19:59.804745 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:19:59.804745 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:19:59.804745 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:19:59.804745 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 06 01:19:59.804745 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:19:59.804745 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 109/214] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Aug 6 01:19:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:19:59.807354 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27545169-36d9-4f77-b280-bf610603b385 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:59.871929 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:59.911166 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36868ed7-93c3-4867-8ed0-db74fe7ef582 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:59.946107 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27545169-36d9-4f77-b280-bf610603b385 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:59.949081 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cff6bb7-67d1-4bb3-9fd0-898f7fcdb9c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:59.949575 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cff6bb7-67d1-4bb3-9fd0-898f7fcdb9c0 None admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:59.949921 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cff6bb7-67d1-4bb3-9fd0-898f7fcdb9c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:59.951136 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cff6bb7-67d1-4bb3-9fd0-898f7fcdb9c0 None admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:59.952033 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cff6bb7-67d1-4bb3-9fd0-898f7fcdb9c0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:59.953916 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cff6bb7-67d1-4bb3-9fd0-898f7fcdb9c0 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:59.983270 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 106/215] 10.222.0.26 () {60 vars in 1132 bytes} [Sat Aug 6 01:19:59 2022] POST /identity/v3/users => generated 297 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:19:59.987344 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1728996a-596e-47c7-b01a-166e18dd3a7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:59.988210 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1728996a-596e-47c7-b01a-166e18dd3a7b None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:59.988732 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1728996a-596e-47c7-b01a-166e18dd3a7b None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:59.989181 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1728996a-596e-47c7-b01a-166e18dd3a7b None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:59.990061 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1728996a-596e-47c7-b01a-166e18dd3a7b None admin] RBAC: Adding query filter params (name=member) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:19:59.990527 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1728996a-596e-47c7-b01a-166e18dd3a7b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:59.992106 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1728996a-596e-47c7-b01a-166e18dd3a7b None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:00.003002 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 110/216] 10.222.0.26 () {56 vars in 1101 bytes} [Sat Aug 6 01:19:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:20:00.009110 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cff6bb7-67d1-4bb3-9fd0-898f7fcdb9c0 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:00.119369 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cff6bb7-67d1-4bb3-9fd0-898f7fcdb9c0 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:00.145633 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:00.157809 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb92fa0-48eb-4ee3-85b9-e10e541a0b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:00.158465 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb92fa0-48eb-4ee3-85b9-e10e541a0b09 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:00.158739 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb92fa0-48eb-4ee3-85b9-e10e541a0b09 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:00.159021 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb92fa0-48eb-4ee3-85b9-e10e541a0b09 None admin] PATH_INFO: `/v3/users/16520a5cd4194361a52b3cf6d37a8053` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:00.175338 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb92fa0-48eb-4ee3-85b9-e10e541a0b09 None admin] RBAC: Authorizing `identity:get_user(user_id=16520a5cd4194361a52b3cf6d37a8053)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:00.176822 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb92fa0-48eb-4ee3-85b9-e10e541a0b09 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:00.178791 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 107/217] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Aug 6 01:20:00 2022] GET /identity/v3/users/16520a5cd4194361a52b3cf6d37a8053 => generated 297 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:00.184178 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1728996a-596e-47c7-b01a-166e18dd3a7b None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:00.194269 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:00.292439 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1728996a-596e-47c7-b01a-166e18dd3a7b None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:00.330954 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0f39bd-5ddb-47a4-babb-5905add47401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:00.331617 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0f39bd-5ddb-47a4-babb-5905add47401 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:00.331905 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0f39bd-5ddb-47a4-babb-5905add47401 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:00.332185 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0f39bd-5ddb-47a4-babb-5905add47401 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:00.417399 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0f39bd-5ddb-47a4-babb-5905add47401 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:00.419705 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0f39bd-5ddb-47a4-babb-5905add47401 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:00.428961 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:00.566120 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 111/218] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Aug 6 01:20:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 01:20:00.572146 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb92fa0-48eb-4ee3-85b9-e10e541a0b09 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:00.676591 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb92fa0-48eb-4ee3-85b9-e10e541a0b09 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:00.718924 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d187c085-f170-4af9-9451-6a3af91a2238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:00.719565 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d187c085-f170-4af9-9451-6a3af91a2238 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:00.719847 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d187c085-f170-4af9-9451-6a3af91a2238 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:00.720125 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d187c085-f170-4af9-9451-6a3af91a2238 None admin] PATH_INFO: `/v3/projects/0bcfa6bfcd464fc9b0dc2079b24902bc` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:00.721340 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d187c085-f170-4af9-9451-6a3af91a2238 None admin] RBAC: Authorizing `identity:get_project(project_id=0bcfa6bfcd464fc9b0dc2079b24902bc)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:00.722778 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d187c085-f170-4af9-9451-6a3af91a2238 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:00.724445 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 108/219] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Aug 6 01:20:00 2022] GET /identity/v3/projects/0bcfa6bfcd464fc9b0dc2079b24902bc => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:00.730756 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0f39bd-5ddb-47a4-babb-5905add47401 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:00.884582 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe95dfd-0e16-4f67-b4ac-f210350e6312 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:00.884885 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe95dfd-0e16-4f67-b4ac-f210350e6312 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:00.885083 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe95dfd-0e16-4f67-b4ac-f210350e6312 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:00.886571 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 109/220] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:20:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:20:00.887468 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0f39bd-5ddb-47a4-babb-5905add47401 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:00.891624 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06b26c9-b711-4450-95d2-4e6ee7acb3b6 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:00.891908 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06b26c9-b711-4450-95d2-4e6ee7acb3b6 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:00.892182 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06b26c9-b711-4450-95d2-4e6ee7acb3b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:00.899695 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:00.924612 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-a06b26c9-b711-4450-95d2-4e6ee7acb3b6 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:20:00.926419 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-a06b26c9-b711-4450-95d2-4e6ee7acb3b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:20:00.931585 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c4118c-988f-4e55-a972-111dcc01267d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:00.932219 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c4118c-988f-4e55-a972-111dcc01267d None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:00.932506 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c4118c-988f-4e55-a972-111dcc01267d None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:00.932786 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c4118c-988f-4e55-a972-111dcc01267d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:00.935222 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c4118c-988f-4e55-a972-111dcc01267d None admin] RBAC: Adding query filter params (role.id=61b3b93d37a047e4bb8369b46fe20db9, scope.project.id=0bcfa6bfcd464fc9b0dc2079b24902bc, user.id=16520a5cd4194361a52b3cf6d37a8053) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:00.935222 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c4118c-988f-4e55-a972-111dcc01267d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:00.935911 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c4118c-988f-4e55-a972-111dcc01267d None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:00.942130 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 112/221] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Aug 6 01:20:00 2022] GET /identity/v3/role_assignments?user.id=16520a5cd4194361a52b3cf6d37a8053&scope.project.id=0bcfa6bfcd464fc9b0dc2079b24902bc&role.id=61b3b93d37a047e4bb8369b46fe20db9 => generated 259 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:01.023462 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:01.048386 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 110/222] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:20:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 01:20:01.088585 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c4118c-988f-4e55-a972-111dcc01267d None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:01.206240 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c4118c-988f-4e55-a972-111dcc01267d None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:01.242797 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92832fbe-40fc-4cb0-9da9-0dfe382fbac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:01.243457 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92832fbe-40fc-4cb0-9da9-0dfe382fbac3 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:01.243742 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92832fbe-40fc-4cb0-9da9-0dfe382fbac3 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:01.244018 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92832fbe-40fc-4cb0-9da9-0dfe382fbac3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:01.258989 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92832fbe-40fc-4cb0-9da9-0dfe382fbac3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:20:01.264692 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-92832fbe-40fc-4cb0-9da9-0dfe382fbac3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:20:01.264692 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:01.264692 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:01.264692 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:01.264692 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:01.264692 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:01.264692 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:01.264692 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:01.264692 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:01.264692 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:01.264692 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:01.264692 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:01.264692 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:20:01.264692 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:20:01.264692 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:20:01.264692 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:20:01.265624 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:20:01.265624 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:20:01.265624 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:20:01.265624 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:20:01.265624 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:01.265624 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:01.265624 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:01.265624 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:01.265624 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:01.265624 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:01.265624 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:01.265624 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:01.265624 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:01.265624 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:01.265624 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:01.265624 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:01.266791 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:01.266791 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:01.266791 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:01.266791 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:01.266791 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:20:01.266791 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:20:01.266791 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:20:01.266791 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:20:01.266791 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:20:01.266791 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:20:01.266791 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:20:01.266791 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:20:01.266791 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 113/223] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Aug 6 01:20:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:20:01.278133 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06b26c9-b711-4450-95d2-4e6ee7acb3b6 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:01.385392 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c044fe4-f43b-4cac-9174-86ae5d437c9b None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:01.385708 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c044fe4-f43b-4cac-9174-86ae5d437c9b None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:01.385964 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c044fe4-f43b-4cac-9174-86ae5d437c9b None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:01.387726 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 114/224] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:20:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:20:01.400646 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a27740c-ca65-4ff6-88e2-5f9dd003e9d9 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:01.400941 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a27740c-ca65-4ff6-88e2-5f9dd003e9d9 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:01.401251 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a27740c-ca65-4ff6-88e2-5f9dd003e9d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:01.422502 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06b26c9-b711-4450-95d2-4e6ee7acb3b6 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:01.426093 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-9a27740c-ca65-4ff6-88e2-5f9dd003e9d9 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:20:01.427298 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-9a27740c-ca65-4ff6-88e2-5f9dd003e9d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:20:01.465493 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:01.495620 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1f3826-f616-4f13-98fb-540d12e682a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:01.496250 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1f3826-f616-4f13-98fb-540d12e682a4 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:01.496536 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1f3826-f616-4f13-98fb-540d12e682a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:01.496813 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1f3826-f616-4f13-98fb-540d12e682a4 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:01.498131 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1f3826-f616-4f13-98fb-540d12e682a4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:01.498440 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1f3826-f616-4f13-98fb-540d12e682a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:01.499813 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1f3826-f616-4f13-98fb-540d12e682a4 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:01.522980 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 111/225] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Aug 6 01:20:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:01.528948 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1f3826-f616-4f13-98fb-540d12e682a4 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:01.545921 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 115/226] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:20:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 01:20:01.569785 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a27740c-ca65-4ff6-88e2-5f9dd003e9d9 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:01.586506 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:01.654996 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1f3826-f616-4f13-98fb-540d12e682a4 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:01.721811 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1a7adc-af66-4bf7-b288-e47de04eedfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:01.722384 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1a7adc-af66-4bf7-b288-e47de04eedfd None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:01.722668 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1a7adc-af66-4bf7-b288-e47de04eedfd None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:01.722948 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1a7adc-af66-4bf7-b288-e47de04eedfd None admin] PATH_INFO: `/v3/users/c1fcb82f07be4ffc85ac133727f3a453` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:01.729783 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1a7adc-af66-4bf7-b288-e47de04eedfd None admin] RBAC: Authorizing `identity:get_user(user_id=c1fcb82f07be4ffc85ac133727f3a453)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:01.731251 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1a7adc-af66-4bf7-b288-e47de04eedfd None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:01.737071 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 112/227] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Aug 6 01:20:01 2022] GET /identity/v3/users/c1fcb82f07be4ffc85ac133727f3a453 => generated 283 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:20:01.750097 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a27740c-ca65-4ff6-88e2-5f9dd003e9d9 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:01.750789 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1a7adc-af66-4bf7-b288-e47de04eedfd None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:01.789814 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6475fbd-440f-4e3d-95ec-a97529d906c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:01.790851 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6475fbd-440f-4e3d-95ec-a97529d906c7 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:01.791331 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6475fbd-440f-4e3d-95ec-a97529d906c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:01.791837 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6475fbd-440f-4e3d-95ec-a97529d906c7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:01.803374 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6475fbd-440f-4e3d-95ec-a97529d906c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:20:01.810404 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-e6475fbd-440f-4e3d-95ec-a97529d906c7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 01:20:01.810404 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:01.810404 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:01.810404 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:01.810404 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:01.810404 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:01.810404 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:01.810404 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:01.810404 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:01.810404 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:01.810404 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:01.810404 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:01.810404 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:20:01.810404 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:20:01.810404 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:20:01.810404 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:20:01.811265 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:20:01.811265 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:20:01.811265 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:20:01.811265 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:20:01.811265 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:01.811265 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:01.811265 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:01.811265 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:01.811265 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:01.811265 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:01.811265 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:01.811265 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:01.811265 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:01.811265 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:01.811265 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:01.811265 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:01.812087 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:01.812087 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:01.812087 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:01.812087 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:01.812087 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:20:01.812087 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:20:01.812087 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:20:01.812087 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:20:01.812087 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:20:01.812087 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:20:01.812087 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 01:20:01.812087 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:20:01.814557 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 116/228] 10.222.0.26 () {56 vars in 1114 bytes} [Sat Aug 6 01:20:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:20:01.820124 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6475fbd-440f-4e3d-95ec-a97529d906c7 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:01.865710 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:01.916104 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1a7adc-af66-4bf7-b288-e47de04eedfd None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:01.928982 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:01.963957 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6188fe-6dc6-4120-8cdf-12a723255b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:01.964610 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6188fe-6dc6-4120-8cdf-12a723255b44 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:01.964896 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6188fe-6dc6-4120-8cdf-12a723255b44 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:01.965211 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6188fe-6dc6-4120-8cdf-12a723255b44 None admin] PATH_INFO: `/v3/projects/0bcfa6bfcd464fc9b0dc2079b24902bc` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:01.968705 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6188fe-6dc6-4120-8cdf-12a723255b44 None admin] RBAC: Authorizing `identity:get_project(project_id=0bcfa6bfcd464fc9b0dc2079b24902bc)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:01.969993 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6188fe-6dc6-4120-8cdf-12a723255b44 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:01.972870 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 113/229] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Aug 6 01:20:01 2022] GET /identity/v3/projects/0bcfa6bfcd464fc9b0dc2079b24902bc => generated 302 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:20:01.979652 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6188fe-6dc6-4120-8cdf-12a723255b44 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:02.043555 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6475fbd-440f-4e3d-95ec-a97529d906c7 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:02.095464 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d5bc05-2a58-438a-9cf3-defd2827d967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:02.096094 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d5bc05-2a58-438a-9cf3-defd2827d967 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:02.096380 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d5bc05-2a58-438a-9cf3-defd2827d967 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:02.096656 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d5bc05-2a58-438a-9cf3-defd2827d967 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:02.097410 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d5bc05-2a58-438a-9cf3-defd2827d967 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:02.097720 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d5bc05-2a58-438a-9cf3-defd2827d967 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:02.101821 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d5bc05-2a58-438a-9cf3-defd2827d967 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:02.120961 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 117/230] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Aug 6 01:20:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:20:02.130725 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d5bc05-2a58-438a-9cf3-defd2827d967 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:02.165627 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6188fe-6dc6-4120-8cdf-12a723255b44 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:02.246058 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:02.247706 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71518799-e8ce-4081-9335-3c7a49e4f662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:02.248628 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71518799-e8ce-4081-9335-3c7a49e4f662 None admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:02.248922 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71518799-e8ce-4081-9335-3c7a49e4f662 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:02.249265 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71518799-e8ce-4081-9335-3c7a49e4f662 None admin] PATH_INFO: `/v3/projects/0bcfa6bfcd464fc9b0dc2079b24902bc/users/c1fcb82f07be4ffc85ac133727f3a453/roles/4559b9a0837249f895a13c70096f8d38` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:02.256866 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71518799-e8ce-4081-9335-3c7a49e4f662 None admin] RBAC: Authorizing `identity:create_grant(project_id=0bcfa6bfcd464fc9b0dc2079b24902bc, user_id=c1fcb82f07be4ffc85ac133727f3a453, role_id=4559b9a0837249f895a13c70096f8d38)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:02.258376 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71518799-e8ce-4081-9335-3c7a49e4f662 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:02.284012 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d5bc05-2a58-438a-9cf3-defd2827d967 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:02.284887 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 114/231] 10.222.0.26 () {58 vars in 1439 bytes} [Sat Aug 6 01:20:01 2022] PUT /identity/v3/projects/0bcfa6bfcd464fc9b0dc2079b24902bc/users/c1fcb82f07be4ffc85ac133727f3a453/roles/4559b9a0837249f895a13c70096f8d38 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:20:02.325437 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ee7948-6fb8-40fc-a371-19f3991432a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:02.326103 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ee7948-6fb8-40fc-a371-19f3991432a8 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:02.326478 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ee7948-6fb8-40fc-a371-19f3991432a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:02.326768 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ee7948-6fb8-40fc-a371-19f3991432a8 None admin] PATH_INFO: `/v3/users/c1fcb82f07be4ffc85ac133727f3a453` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:02.328039 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ee7948-6fb8-40fc-a371-19f3991432a8 None admin] RBAC: Authorizing `identity:get_user(user_id=c1fcb82f07be4ffc85ac133727f3a453)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:02.329625 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ee7948-6fb8-40fc-a371-19f3991432a8 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:02.331759 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 118/232] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Aug 6 01:20:02 2022] GET /identity/v3/users/c1fcb82f07be4ffc85ac133727f3a453 => generated 283 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:20:02.337356 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71518799-e8ce-4081-9335-3c7a49e4f662 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:02.385455 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:02.453489 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71518799-e8ce-4081-9335-3c7a49e4f662 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:02.491209 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307269c3-a010-460e-8e60-c8890983fdf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:02.492161 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307269c3-a010-460e-8e60-c8890983fdf5 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:02.492444 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307269c3-a010-460e-8e60-c8890983fdf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:02.492723 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307269c3-a010-460e-8e60-c8890983fdf5 None admin] PATH_INFO: `/v3/projects/0bcfa6bfcd464fc9b0dc2079b24902bc` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:02.493947 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307269c3-a010-460e-8e60-c8890983fdf5 None admin] RBAC: Authorizing `identity:get_project(project_id=0bcfa6bfcd464fc9b0dc2079b24902bc)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:02.495411 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307269c3-a010-460e-8e60-c8890983fdf5 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:02.497855 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 115/233] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Aug 6 01:20:02 2022] GET /identity/v3/projects/0bcfa6bfcd464fc9b0dc2079b24902bc => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:02.503904 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ee7948-6fb8-40fc-a371-19f3991432a8 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:02.615119 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ee7948-6fb8-40fc-a371-19f3991432a8 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:02.642355 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9416f479-15d2-4aa3-9bae-a1bce22015de None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:02.642682 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9416f479-15d2-4aa3-9bae-a1bce22015de None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:02.642930 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9416f479-15d2-4aa3-9bae-a1bce22015de None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:02.644417 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 116/234] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:20:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:20:02.649574 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea66dea5-571a-48b8-8154-f0a330edcc0a None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:02.649863 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea66dea5-571a-48b8-8154-f0a330edcc0a None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:02.650143 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea66dea5-571a-48b8-8154-f0a330edcc0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:02.657652 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111c9da2-beb7-4bb9-9281-cf893c8a6b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:02.658636 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111c9da2-beb7-4bb9-9281-cf893c8a6b09 None admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:02.659348 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111c9da2-beb7-4bb9-9281-cf893c8a6b09 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:02.659768 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111c9da2-beb7-4bb9-9281-cf893c8a6b09 None admin] PATH_INFO: `/v3/projects/0bcfa6bfcd464fc9b0dc2079b24902bc/users/c1fcb82f07be4ffc85ac133727f3a453/roles/6d8b63d350944d78812edbefb32ae8ee` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:02.663699 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111c9da2-beb7-4bb9-9281-cf893c8a6b09 None admin] RBAC: Authorizing `identity:create_grant(project_id=0bcfa6bfcd464fc9b0dc2079b24902bc, user_id=c1fcb82f07be4ffc85ac133727f3a453, role_id=6d8b63d350944d78812edbefb32ae8ee)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:02.665362 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111c9da2-beb7-4bb9-9281-cf893c8a6b09 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:02.677082 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-ea66dea5-571a-48b8-8154-f0a330edcc0a None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:20:02.678600 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-ea66dea5-571a-48b8-8154-f0a330edcc0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:20:02.681110 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 119/235] 10.222.0.26 () {58 vars in 1439 bytes} [Sat Aug 6 01:20:02 2022] PUT /identity/v3/projects/0bcfa6bfcd464fc9b0dc2079b24902bc/users/c1fcb82f07be4ffc85ac133727f3a453/roles/6d8b63d350944d78812edbefb32ae8ee => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:20:02.766770 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 117/236] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:20:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 01:20:02.807600 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111c9da2-beb7-4bb9-9281-cf893c8a6b09 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:02.828724 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287be6f6-d814-452e-ab2e-00823201f98c None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:02.829014 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287be6f6-d814-452e-ab2e-00823201f98c None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:02.829334 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287be6f6-d814-452e-ab2e-00823201f98c None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:02.830736 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 118/237] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:20:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:20:02.838816 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c487afb4-6e52-4452-82ff-669c5f9a1d79 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:02.839305 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:02.839632 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c487afb4-6e52-4452-82ff-669c5f9a1d79 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:02.840096 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c487afb4-6e52-4452-82ff-669c5f9a1d79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:02.875042 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-c487afb4-6e52-4452-82ff-669c5f9a1d79 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:20:02.876244 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-c487afb4-6e52-4452-82ff-669c5f9a1d79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:20:02.929425 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:02.940857 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111c9da2-beb7-4bb9-9281-cf893c8a6b09 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:02.955835 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 119/238] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:20:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:20:02.982320 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9b9c50-0c43-47e8-a7f3-72d55321f006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:02.983080 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9b9c50-0c43-47e8-a7f3-72d55321f006 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:02.983559 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9b9c50-0c43-47e8-a7f3-72d55321f006 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:02.983662 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9b9c50-0c43-47e8-a7f3-72d55321f006 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:02.984649 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c487afb4-6e52-4452-82ff-669c5f9a1d79 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:02.994953 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9b9c50-0c43-47e8-a7f3-72d55321f006 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:20:03.001320 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-9c9b9c50-0c43-47e8-a7f3-72d55321f006 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 06 01:20:03.001320 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:03.001320 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:03.001320 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:03.001320 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:03.001320 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:03.001320 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:03.001320 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:03.001320 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:03.001320 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:03.001320 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:03.001320 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:03.001320 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:20:03.001320 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:20:03.001320 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:20:03.001320 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:20:03.002185 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:20:03.002185 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:20:03.002185 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:20:03.002185 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:20:03.002185 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:03.002185 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:03.002185 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:03.002185 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:03.002185 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:03.002185 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:03.002185 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:03.002185 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:03.002185 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:03.002185 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:03.002185 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:03.002185 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:03.002980 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:03.002980 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:03.002980 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:03.002980 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:03.002980 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:20:03.002980 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:20:03.002980 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:20:03.002980 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:20:03.002980 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:20:03.002980 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:20:03.002980 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 06 01:20:03.002980 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:20:03.002980 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 120/239] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Aug 6 01:20:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:20:03.008364 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9b9c50-0c43-47e8-a7f3-72d55321f006 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:03.093454 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:03.127726 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9b9c50-0c43-47e8-a7f3-72d55321f006 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:03.172748 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01351821-4631-4c2f-8fef-a544e0f37cf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:03.173539 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01351821-4631-4c2f-8fef-a544e0f37cf2 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:03.173825 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01351821-4631-4c2f-8fef-a544e0f37cf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:03.174104 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01351821-4631-4c2f-8fef-a544e0f37cf2 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:03.174824 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01351821-4631-4c2f-8fef-a544e0f37cf2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:03.175130 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01351821-4631-4c2f-8fef-a544e0f37cf2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:03.180541 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01351821-4631-4c2f-8fef-a544e0f37cf2 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:03.205414 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 121/240] 10.222.0.26 () {56 vars in 1101 bytes} [Sat Aug 6 01:20:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:20:03.206416 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01351821-4631-4c2f-8fef-a544e0f37cf2 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:03.237019 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c487afb4-6e52-4452-82ff-669c5f9a1d79 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:03.316837 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e5ee86-088c-413c-8ff6-d63f54c38d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:03.317529 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e5ee86-088c-413c-8ff6-d63f54c38d5f None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:03.317842 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e5ee86-088c-413c-8ff6-d63f54c38d5f None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:03.318116 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e5ee86-088c-413c-8ff6-d63f54c38d5f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:03.320402 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e5ee86-088c-413c-8ff6-d63f54c38d5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:03.321884 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e5ee86-088c-413c-8ff6-d63f54c38d5f None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:03.323660 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 120/241] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:20:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:20:03.329197 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e5ee86-088c-413c-8ff6-d63f54c38d5f None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:03.381629 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01351821-4631-4c2f-8fef-a544e0f37cf2 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:03.387406 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:03.423355 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:03.432268 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37014f4-9dd6-4b7c-a20d-47bdff87ceeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:03.434003 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37014f4-9dd6-4b7c-a20d-47bdff87ceeb None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:03.434645 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37014f4-9dd6-4b7c-a20d-47bdff87ceeb None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:03.437409 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37014f4-9dd6-4b7c-a20d-47bdff87ceeb None admin] PATH_INFO: `/v3/users/ac3fe351cf5b49c395af5d46bc833105` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:03.460161 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37014f4-9dd6-4b7c-a20d-47bdff87ceeb None admin] RBAC: Authorizing `identity:get_user(user_id=ac3fe351cf5b49c395af5d46bc833105)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:03.462561 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37014f4-9dd6-4b7c-a20d-47bdff87ceeb None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:03.469124 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 122/242] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Aug 6 01:20:03 2022] GET /identity/v3/users/ac3fe351cf5b49c395af5d46bc833105 => generated 297 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:03.474692 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37014f4-9dd6-4b7c-a20d-47bdff87ceeb None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:03.521241 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e5ee86-088c-413c-8ff6-d63f54c38d5f None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:03.556857 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af2f48c-a535-4843-8601-6d578ff79db8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:03.557517 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af2f48c-a535-4843-8601-6d578ff79db8 None admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:03.557801 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af2f48c-a535-4843-8601-6d578ff79db8 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:03.558090 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af2f48c-a535-4843-8601-6d578ff79db8 None admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:03.558854 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af2f48c-a535-4843-8601-6d578ff79db8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:03.560961 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af2f48c-a535-4843-8601-6d578ff79db8 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:03.587977 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37014f4-9dd6-4b7c-a20d-47bdff87ceeb None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:03.593500 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 121/243] 10.222.0.26 () {60 vars in 1132 bytes} [Sat Aug 6 01:20:03 2022] POST /identity/v3/users => generated 293 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:20:03.595053 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2c6c34-b697-470d-8b06-5d56777ac8cf None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:03.601637 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2c6c34-b697-470d-8b06-5d56777ac8cf None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:03.605089 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2c6c34-b697-470d-8b06-5d56777ac8cf None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:03.605089 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 122/244] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:20:03 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:20:03.614928 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f404ec88-cfd0-4f4c-862e-219117e0d005 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:03.615233 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f404ec88-cfd0-4f4c-862e-219117e0d005 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:03.619446 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f404ec88-cfd0-4f4c-862e-219117e0d005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:03.650328 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-f404ec88-cfd0-4f4c-862e-219117e0d005 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:20:03.651482 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-f404ec88-cfd0-4f4c-862e-219117e0d005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:20:03.705339 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c68351-e04b-4939-bf9d-91154d1deefd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:03.706049 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c68351-e04b-4939-bf9d-91154d1deefd None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:03.706341 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c68351-e04b-4939-bf9d-91154d1deefd None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:03.706622 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c68351-e04b-4939-bf9d-91154d1deefd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:03.742996 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 123/245] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:20:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:20:03.777588 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f404ec88-cfd0-4f4c-862e-219117e0d005 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:03.789214 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:03.802757 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c68351-e04b-4939-bf9d-91154d1deefd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:03.804106 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c68351-e04b-4939-bf9d-91154d1deefd None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:03.806240 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:03.937911 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f404ec88-cfd0-4f4c-862e-219117e0d005 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:03.943706 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 123/246] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Aug 6 01:20:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 01:20:03.950024 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c68351-e04b-4939-bf9d-91154d1deefd None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:03.995222 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884bcb22-6495-4424-b3b3-df344f176996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:03.995884 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884bcb22-6495-4424-b3b3-df344f176996 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:03.996163 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884bcb22-6495-4424-b3b3-df344f176996 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:03.996437 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884bcb22-6495-4424-b3b3-df344f176996 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:04.014489 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884bcb22-6495-4424-b3b3-df344f176996 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:20:04.019645 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-884bcb22-6495-4424-b3b3-df344f176996 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 06 01:20:04.019645 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:04.019645 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:04.019645 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:04.019645 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:04.019645 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:04.019645 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:04.019645 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:04.019645 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:04.019645 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:04.019645 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:04.019645 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:04.019645 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:20:04.019645 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:20:04.019645 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:20:04.019645 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:20:04.020579 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:20:04.020579 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:20:04.020579 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:20:04.020579 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:20:04.020579 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:04.020579 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:04.020579 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:04.020579 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:04.020579 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:04.020579 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:04.020579 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:04.020579 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:04.020579 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:04.020579 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:04.020579 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:04.020579 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:04.021394 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:04.021394 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:04.021394 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:04.021394 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:04.021394 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:20:04.021394 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:20:04.021394 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:20:04.021394 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:20:04.021394 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:20:04.021394 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:20:04.021394 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 06 01:20:04.021394 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:20:04.021394 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 124/247] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Aug 6 01:20:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:20:04.026902 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884bcb22-6495-4424-b3b3-df344f176996 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:04.078459 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:04.091683 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c68351-e04b-4939-bf9d-91154d1deefd None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:04.139952 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:04.142262 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a214f32e-687f-4766-a968-bc87458703cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:04.142923 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a214f32e-687f-4766-a968-bc87458703cc None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:04.143238 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a214f32e-687f-4766-a968-bc87458703cc None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:04.143541 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a214f32e-687f-4766-a968-bc87458703cc None admin] PATH_INFO: `/v3/projects/0bcfa6bfcd464fc9b0dc2079b24902bc` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:04.144746 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a214f32e-687f-4766-a968-bc87458703cc None admin] RBAC: Authorizing `identity:get_project(project_id=0bcfa6bfcd464fc9b0dc2079b24902bc)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:04.146259 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a214f32e-687f-4766-a968-bc87458703cc None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:04.148334 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 124/248] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Aug 6 01:20:03 2022] GET /identity/v3/projects/0bcfa6bfcd464fc9b0dc2079b24902bc => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:04.154172 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a214f32e-687f-4766-a968-bc87458703cc None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:04.163621 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884bcb22-6495-4424-b3b3-df344f176996 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:04.205971 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddd44a7-10a4-4d39-87c9-77fa72fde396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:04.206530 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddd44a7-10a4-4d39-87c9-77fa72fde396 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:04.206813 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddd44a7-10a4-4d39-87c9-77fa72fde396 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:04.207088 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddd44a7-10a4-4d39-87c9-77fa72fde396 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:04.207817 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddd44a7-10a4-4d39-87c9-77fa72fde396 None admin] RBAC: Adding query filter params (name=member) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:04.208120 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddd44a7-10a4-4d39-87c9-77fa72fde396 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:04.209548 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddd44a7-10a4-4d39-87c9-77fa72fde396 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:04.220191 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 125/249] 10.222.0.26 () {56 vars in 1101 bytes} [Sat Aug 6 01:20:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:04.225913 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddd44a7-10a4-4d39-87c9-77fa72fde396 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:04.280900 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a214f32e-687f-4766-a968-bc87458703cc None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:04.319357 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75016c6-cf09-4a86-8c6f-e7d0066e482c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:04.319991 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75016c6-cf09-4a86-8c6f-e7d0066e482c None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:04.320367 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75016c6-cf09-4a86-8c6f-e7d0066e482c None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:04.320649 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75016c6-cf09-4a86-8c6f-e7d0066e482c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:04.321478 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75016c6-cf09-4a86-8c6f-e7d0066e482c None admin] RBAC: Adding query filter params (role.id=25efb2aed94e4d63b0b6cb8faf5a2ee7, scope.project.id=0bcfa6bfcd464fc9b0dc2079b24902bc, user.id=ac3fe351cf5b49c395af5d46bc833105) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:04.321791 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75016c6-cf09-4a86-8c6f-e7d0066e482c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:04.323469 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75016c6-cf09-4a86-8c6f-e7d0066e482c None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:04.330233 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 125/250] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Aug 6 01:20:04 2022] GET /identity/v3/role_assignments?user.id=ac3fe351cf5b49c395af5d46bc833105&scope.project.id=0bcfa6bfcd464fc9b0dc2079b24902bc&role.id=25efb2aed94e4d63b0b6cb8faf5a2ee7 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:20:04.335686 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddd44a7-10a4-4d39-87c9-77fa72fde396 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:04.375708 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725b1720-888a-486e-bc3a-d07ece068e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:04.376442 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725b1720-888a-486e-bc3a-d07ece068e05 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:04.376727 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725b1720-888a-486e-bc3a-d07ece068e05 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:04.377008 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725b1720-888a-486e-bc3a-d07ece068e05 None admin] PATH_INFO: `/v3/users/16520a5cd4194361a52b3cf6d37a8053` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:04.378271 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725b1720-888a-486e-bc3a-d07ece068e05 None admin] RBAC: Authorizing `identity:get_user(user_id=16520a5cd4194361a52b3cf6d37a8053)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:04.379739 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725b1720-888a-486e-bc3a-d07ece068e05 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:04.381429 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 126/251] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Aug 6 01:20:04 2022] GET /identity/v3/users/16520a5cd4194361a52b3cf6d37a8053 => generated 297 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:20:04.387040 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75016c6-cf09-4a86-8c6f-e7d0066e482c None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:04.429646 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:04.538600 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75016c6-cf09-4a86-8c6f-e7d0066e482c None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:04.576858 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1a94bb-fd34-4d7e-aaeb-f3b2db72597f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:04.577559 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1a94bb-fd34-4d7e-aaeb-f3b2db72597f None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:04.577843 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1a94bb-fd34-4d7e-aaeb-f3b2db72597f None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:04.578123 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1a94bb-fd34-4d7e-aaeb-f3b2db72597f None admin] PATH_INFO: `/v3/projects/0bcfa6bfcd464fc9b0dc2079b24902bc` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:04.579372 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1a94bb-fd34-4d7e-aaeb-f3b2db72597f None admin] RBAC: Authorizing `identity:get_project(project_id=0bcfa6bfcd464fc9b0dc2079b24902bc)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:04.580822 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1a94bb-fd34-4d7e-aaeb-f3b2db72597f None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:04.582743 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 126/252] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Aug 6 01:20:04 2022] GET /identity/v3/projects/0bcfa6bfcd464fc9b0dc2079b24902bc => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:04.588824 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725b1720-888a-486e-bc3a-d07ece068e05 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:04.609095 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:04.717813 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725b1720-888a-486e-bc3a-d07ece068e05 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:04.756648 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d70cc7-b5ee-4d97-86f6-761aab9f8a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:04.757777 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d70cc7-b5ee-4d97-86f6-761aab9f8a39 None admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:04.758259 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d70cc7-b5ee-4d97-86f6-761aab9f8a39 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:04.758730 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d70cc7-b5ee-4d97-86f6-761aab9f8a39 None admin] PATH_INFO: `/v3/projects/0bcfa6bfcd464fc9b0dc2079b24902bc/users/16520a5cd4194361a52b3cf6d37a8053/roles/61b3b93d37a047e4bb8369b46fe20db9` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:04.760907 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d70cc7-b5ee-4d97-86f6-761aab9f8a39 None admin] RBAC: Authorizing `identity:create_grant(project_id=0bcfa6bfcd464fc9b0dc2079b24902bc, user_id=16520a5cd4194361a52b3cf6d37a8053, role_id=61b3b93d37a047e4bb8369b46fe20db9)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:04.762688 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d70cc7-b5ee-4d97-86f6-761aab9f8a39 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:04.784016 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 127/253] 10.222.0.26 () {58 vars in 1439 bytes} [Sat Aug 6 01:20:04 2022] PUT /identity/v3/projects/0bcfa6bfcd464fc9b0dc2079b24902bc/users/16520a5cd4194361a52b3cf6d37a8053/roles/61b3b93d37a047e4bb8369b46fe20db9 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:20:05.148575 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477eee9e-d726-4ee6-992d-8723c46d1891 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:05.149137 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477eee9e-d726-4ee6-992d-8723c46d1891 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:05.149137 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477eee9e-d726-4ee6-992d-8723c46d1891 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:05.150646 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 127/254] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:20:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:20:05.157312 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8480d900-c098-4db6-be4e-1ec77a774113 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:05.157925 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8480d900-c098-4db6-be4e-1ec77a774113 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:05.158399 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8480d900-c098-4db6-be4e-1ec77a774113 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:05.178472 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-8480d900-c098-4db6-be4e-1ec77a774113 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:20:05.179994 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-8480d900-c098-4db6-be4e-1ec77a774113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:20:05.247049 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:05.257185 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 128/255] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:20:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:20:05.275871 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477eee9e-d726-4ee6-992d-8723c46d1891 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:05.374573 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:05.385807 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477eee9e-d726-4ee6-992d-8723c46d1891 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:05.443150 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0625f70-7925-4b4d-bfc1-7831ea0ea3b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:05.443807 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0625f70-7925-4b4d-bfc1-7831ea0ea3b8 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:05.444090 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0625f70-7925-4b4d-bfc1-7831ea0ea3b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:05.444369 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0625f70-7925-4b4d-bfc1-7831ea0ea3b8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:05.458037 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0625f70-7925-4b4d-bfc1-7831ea0ea3b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:20:05.463410 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-c0625f70-7925-4b4d-bfc1-7831ea0ea3b8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:20:05.463410 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:05.463410 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:05.463410 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:05.463410 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:05.463410 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:05.463410 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:05.463410 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:05.463410 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:05.463410 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:05.463410 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:05.463410 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:05.463410 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:20:05.463410 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:20:05.463410 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:20:05.463410 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:20:05.464259 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:20:05.464259 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:20:05.464259 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:20:05.464259 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:20:05.464259 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:05.464259 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:05.464259 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:05.464259 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:05.464259 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:05.464259 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:05.464259 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:05.464259 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:05.464259 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:05.464259 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:05.464259 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:05.464259 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:05.465067 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:05.465067 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:05.465067 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:05.465067 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:05.465067 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:20:05.465067 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:20:05.465067 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:20:05.465067 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:20:05.465067 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:20:05.465067 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:20:05.465067 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:20:05.465067 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:20:05.469139 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 128/256] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Aug 6 01:20:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:20:05.470520 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8480d900-c098-4db6-be4e-1ec77a774113 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:05.642110 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8480d900-c098-4db6-be4e-1ec77a774113 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:05.693874 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2ef782-c44c-4221-877e-521a63c6893c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:05.694844 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2ef782-c44c-4221-877e-521a63c6893c None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:05.695341 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2ef782-c44c-4221-877e-521a63c6893c None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:05.695809 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2ef782-c44c-4221-877e-521a63c6893c None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:05.696748 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2ef782-c44c-4221-877e-521a63c6893c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:05.697269 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2ef782-c44c-4221-877e-521a63c6893c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:05.698852 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2ef782-c44c-4221-877e-521a63c6893c None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:05.712842 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 129/257] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Aug 6 01:20:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:05.719323 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0625f70-7925-4b4d-bfc1-7831ea0ea3b8 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:05.774873 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23a181d-1e40-4f44-ac8f-719b00540de7 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:05.775521 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23a181d-1e40-4f44-ac8f-719b00540de7 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:05.775993 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23a181d-1e40-4f44-ac8f-719b00540de7 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:05.777942 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 130/258] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:20:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:20:05.782801 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc631669-23f3-492b-a665-642e02a5672b None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:05.783332 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc631669-23f3-492b-a665-642e02a5672b None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:05.783885 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc631669-23f3-492b-a665-642e02a5672b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:05.818051 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-cc631669-23f3-492b-a665-642e02a5672b None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:20:05.819675 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-cc631669-23f3-492b-a665-642e02a5672b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:20:05.845452 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0625f70-7925-4b4d-bfc1-7831ea0ea3b8 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:05.881885 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e89d76-77ac-4394-ae59-fe1233c7a69f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:05.882535 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e89d76-77ac-4394-ae59-fe1233c7a69f None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:05.882820 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e89d76-77ac-4394-ae59-fe1233c7a69f None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:05.883095 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e89d76-77ac-4394-ae59-fe1233c7a69f None admin] PATH_INFO: `/v3/users/c1fcb82f07be4ffc85ac133727f3a453` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:05.884296 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e89d76-77ac-4394-ae59-fe1233c7a69f None admin] RBAC: Authorizing `identity:get_user(user_id=c1fcb82f07be4ffc85ac133727f3a453)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:05.885805 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e89d76-77ac-4394-ae59-fe1233c7a69f None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:05.888461 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 129/259] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Aug 6 01:20:05 2022] GET /identity/v3/users/c1fcb82f07be4ffc85ac133727f3a453 => generated 283 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:05.894625 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e89d76-77ac-4394-ae59-fe1233c7a69f None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:05.941455 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:05.972471 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:06.018357 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 131/260] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:20:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 01:20:06.037318 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e89d76-77ac-4394-ae59-fe1233c7a69f None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:06.058612 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc631669-23f3-492b-a665-642e02a5672b None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:06.084040 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dd0e24-a5dd-4a8f-8d4f-475cd83f284e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:06.084744 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dd0e24-a5dd-4a8f-8d4f-475cd83f284e None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:06.085033 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dd0e24-a5dd-4a8f-8d4f-475cd83f284e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:06.085382 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dd0e24-a5dd-4a8f-8d4f-475cd83f284e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:06.229782 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dd0e24-a5dd-4a8f-8d4f-475cd83f284e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:06.232683 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dd0e24-a5dd-4a8f-8d4f-475cd83f284e None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:06.273865 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:06.279757 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc631669-23f3-492b-a665-642e02a5672b None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:06.359332 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 130/261] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Aug 6 01:20:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 01:20:06.367315 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dd0e24-a5dd-4a8f-8d4f-475cd83f284e None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:06.417001 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba37231b-ad16-4641-b5fd-2492654b6dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:06.418018 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba37231b-ad16-4641-b5fd-2492654b6dae None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:06.418514 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba37231b-ad16-4641-b5fd-2492654b6dae None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:06.419563 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba37231b-ad16-4641-b5fd-2492654b6dae None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:06.453961 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba37231b-ad16-4641-b5fd-2492654b6dae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:20:06.459277 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-ba37231b-ad16-4641-b5fd-2492654b6dae None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 01:20:06.459277 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:06.459277 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:06.459277 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:06.459277 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:06.459277 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:06.459277 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:06.459277 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:06.459277 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:06.459277 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:06.459277 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:06.459277 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:06.459277 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:20:06.459277 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:20:06.459277 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:20:06.459277 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:20:06.460127 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:20:06.460127 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:20:06.460127 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:20:06.460127 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:20:06.460127 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:06.460127 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:06.460127 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:06.460127 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:06.460127 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:06.460127 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:06.460127 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:06.460127 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:06.460127 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:06.460127 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:06.460127 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:06.460127 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:06.460906 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:06.460906 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:06.460906 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:06.460906 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:06.460906 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:20:06.460906 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:20:06.460906 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:20:06.460906 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:20:06.460906 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:20:06.460906 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:20:06.460906 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 01:20:06.460906 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:20:06.463218 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 132/262] 10.222.0.26 () {56 vars in 1114 bytes} [Sat Aug 6 01:20:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 405 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:20:06.469641 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba37231b-ad16-4641-b5fd-2492654b6dae None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:06.588764 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:06.589924 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dd0e24-a5dd-4a8f-8d4f-475cd83f284e None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:06.625358 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:06.659966 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0a4478-559f-4486-8f75-6075cd2aff07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:06.660627 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0a4478-559f-4486-8f75-6075cd2aff07 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:06.660914 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0a4478-559f-4486-8f75-6075cd2aff07 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:06.661226 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0a4478-559f-4486-8f75-6075cd2aff07 None admin] PATH_INFO: `/v3/projects/0bcfa6bfcd464fc9b0dc2079b24902bc` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:06.663267 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0a4478-559f-4486-8f75-6075cd2aff07 None admin] RBAC: Authorizing `identity:get_project(project_id=0bcfa6bfcd464fc9b0dc2079b24902bc)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:06.664743 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0a4478-559f-4486-8f75-6075cd2aff07 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:06.668559 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 131/263] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Aug 6 01:20:06 2022] GET /identity/v3/projects/0bcfa6bfcd464fc9b0dc2079b24902bc => generated 302 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:06.674124 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0a4478-559f-4486-8f75-6075cd2aff07 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:06.696890 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba37231b-ad16-4641-b5fd-2492654b6dae None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:06.763046 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeed9aa4-4dd7-4ec0-bb33-1ba7c4e6345a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:06.763844 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeed9aa4-4dd7-4ec0-bb33-1ba7c4e6345a None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:06.764130 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeed9aa4-4dd7-4ec0-bb33-1ba7c4e6345a None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:06.764408 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeed9aa4-4dd7-4ec0-bb33-1ba7c4e6345a None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:06.765161 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeed9aa4-4dd7-4ec0-bb33-1ba7c4e6345a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:06.765471 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeed9aa4-4dd7-4ec0-bb33-1ba7c4e6345a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:06.766861 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeed9aa4-4dd7-4ec0-bb33-1ba7c4e6345a None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:06.783654 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 133/264] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Aug 6 01:20:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:06.789122 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeed9aa4-4dd7-4ec0-bb33-1ba7c4e6345a None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:06.901200 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0a4478-559f-4486-8f75-6075cd2aff07 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:06.990782 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cb9922-cb35-4ff3-9a39-9b95e16e5cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:06.991421 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cb9922-cb35-4ff3-9a39-9b95e16e5cd5 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:06.991707 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cb9922-cb35-4ff3-9a39-9b95e16e5cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:06.991991 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cb9922-cb35-4ff3-9a39-9b95e16e5cd5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:06.992795 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cb9922-cb35-4ff3-9a39-9b95e16e5cd5 None admin] RBAC: Adding query filter params (role.id=4559b9a0837249f895a13c70096f8d38, scope.project.id=0bcfa6bfcd464fc9b0dc2079b24902bc, user.id=c1fcb82f07be4ffc85ac133727f3a453) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:06.993135 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cb9922-cb35-4ff3-9a39-9b95e16e5cd5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:06.994578 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cb9922-cb35-4ff3-9a39-9b95e16e5cd5 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:07.008568 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeed9aa4-4dd7-4ec0-bb33-1ba7c4e6345a None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:07.015081 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 132/265] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Aug 6 01:20:06 2022] GET /identity/v3/role_assignments?user.id=c1fcb82f07be4ffc85ac133727f3a453&scope.project.id=0bcfa6bfcd464fc9b0dc2079b24902bc&role.id=4559b9a0837249f895a13c70096f8d38 => generated 608 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:07.016359 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997ae7bf-ff9c-4295-9bad-d08cfb0f89a0 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:07.016653 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997ae7bf-ff9c-4295-9bad-d08cfb0f89a0 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:07.016932 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997ae7bf-ff9c-4295-9bad-d08cfb0f89a0 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:07.018190 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 133/266] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:20:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:20:07.036244 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:07.038588 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c989f7c4-0a97-4a5e-9637-67bb6df85c37 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:07.038876 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c989f7c4-0a97-4a5e-9637-67bb6df85c37 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:07.039160 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c989f7c4-0a97-4a5e-9637-67bb6df85c37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:07.064305 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e27ba16-bd22-4469-9ab2-02a89974b294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:07.065133 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e27ba16-bd22-4469-9ab2-02a89974b294 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:07.065423 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e27ba16-bd22-4469-9ab2-02a89974b294 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:07.065704 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e27ba16-bd22-4469-9ab2-02a89974b294 None admin] PATH_INFO: `/v3/users/c1fcb82f07be4ffc85ac133727f3a453` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:07.066966 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e27ba16-bd22-4469-9ab2-02a89974b294 None admin] RBAC: Authorizing `identity:get_user(user_id=c1fcb82f07be4ffc85ac133727f3a453)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:07.069149 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e27ba16-bd22-4469-9ab2-02a89974b294 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:07.071157 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 134/267] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Aug 6 01:20:06 2022] GET /identity/v3/users/c1fcb82f07be4ffc85ac133727f3a453 => generated 283 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:07.078138 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e27ba16-bd22-4469-9ab2-02a89974b294 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:07.079957 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-c989f7c4-0a97-4a5e-9637-67bb6df85c37 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:20:07.081259 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-c989f7c4-0a97-4a5e-9637-67bb6df85c37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:20:07.205247 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e27ba16-bd22-4469-9ab2-02a89974b294 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:07.213446 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:07.225389 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 134/268] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:20:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:20:07.244846 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acd70f0-4a66-4cc4-9f28-e43d334f1385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:07.245504 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acd70f0-4a66-4cc4-9f28-e43d334f1385 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:07.245956 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acd70f0-4a66-4cc4-9f28-e43d334f1385 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:07.246385 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acd70f0-4a66-4cc4-9f28-e43d334f1385 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:07.254138 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c989f7c4-0a97-4a5e-9637-67bb6df85c37 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:07.313850 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:07.337235 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acd70f0-4a66-4cc4-9f28-e43d334f1385 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:07.339441 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acd70f0-4a66-4cc4-9f28-e43d334f1385 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:07.415824 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c989f7c4-0a97-4a5e-9637-67bb6df85c37 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:07.455928 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9455ef-0b59-43c5-93e5-1dde81f040fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:07.456501 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9455ef-0b59-43c5-93e5-1dde81f040fb None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:07.456786 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9455ef-0b59-43c5-93e5-1dde81f040fb None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:07.457349 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9455ef-0b59-43c5-93e5-1dde81f040fb None admin] PATH_INFO: `/v3/roles/member` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:07.469165 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9455ef-0b59-43c5-93e5-1dde81f040fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:20:07.470004 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 135/269] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Aug 6 01:20:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 01:20:07.471261 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042186b7-d364-4839-bc7f-cc1385eecfe3 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:07.471546 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042186b7-d364-4839-bc7f-cc1385eecfe3 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:07.471820 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042186b7-d364-4839-bc7f-cc1385eecfe3 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:07.477462 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 136/270] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:20:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:20:07.477462 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a135aa-6492-4bdd-aba8-532411021c6e None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:07.477462 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a135aa-6492-4bdd-aba8-532411021c6e None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:07.477462 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a135aa-6492-4bdd-aba8-532411021c6e None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:07.477462 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 137/271] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:20:07 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:20:07.480294 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-ce9455ef-0b59-43c5-93e5-1dde81f040fb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 06 01:20:07.480294 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:07.480294 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:07.480294 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:07.480294 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:07.480294 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:07.480294 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:07.480294 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:07.480294 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:07.480294 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:07.480294 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:07.480294 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:07.480294 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:20:07.480294 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:20:07.480294 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:20:07.480294 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:20:07.481170 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:20:07.481170 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:20:07.481170 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:20:07.481170 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:20:07.481170 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:07.481170 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:07.481170 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:07.481170 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:07.481170 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:07.481170 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:07.481170 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:07.481170 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:07.481170 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:07.481170 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:07.481170 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:07.481170 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:07.482171 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:07.482171 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:07.482171 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:07.482171 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:07.482171 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:20:07.482171 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:20:07.482171 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:20:07.482171 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:20:07.482171 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:20:07.482171 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:20:07.482171 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 06 01:20:07.482171 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:20:07.482762 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a135aa-6492-4bdd-aba8-532411021c6e None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:07.482831 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 135/272] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Aug 6 01:20:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:20:07.482895 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa409ee-0c38-4dd9-9121-53147640cca8 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:07.483539 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa409ee-0c38-4dd9-9121-53147640cca8 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:07.483539 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa409ee-0c38-4dd9-9121-53147640cca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:07.505611 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-5aa409ee-0c38-4dd9-9121-53147640cca8 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:20:07.506792 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-5aa409ee-0c38-4dd9-9121-53147640cca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:20:07.542190 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:07.577226 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 136/273] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:20:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:20:07.577826 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:07.578625 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17eed5c-9d3c-4404-af37-9ad501b03c08 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:07.578921 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17eed5c-9d3c-4404-af37-9ad501b03c08 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:07.579199 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17eed5c-9d3c-4404-af37-9ad501b03c08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:07.602810 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-d17eed5c-9d3c-4404-af37-9ad501b03c08 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:20:07.604013 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-d17eed5c-9d3c-4404-af37-9ad501b03c08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:20:07.604663 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a135aa-6492-4bdd-aba8-532411021c6e None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:07.642329 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d59f4a-3745-401a-a542-cd76b1e75e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:07.643674 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d59f4a-3745-401a-a542-cd76b1e75e3f None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:07.643674 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d59f4a-3745-401a-a542-cd76b1e75e3f None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:07.643674 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d59f4a-3745-401a-a542-cd76b1e75e3f None admin] PATH_INFO: `/v3/projects/0bcfa6bfcd464fc9b0dc2079b24902bc` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:07.644655 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d59f4a-3745-401a-a542-cd76b1e75e3f None admin] RBAC: Authorizing `identity:get_project(project_id=0bcfa6bfcd464fc9b0dc2079b24902bc)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:07.646141 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d59f4a-3745-401a-a542-cd76b1e75e3f None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:07.647972 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 138/274] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Aug 6 01:20:07 2022] GET /identity/v3/projects/0bcfa6bfcd464fc9b0dc2079b24902bc => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:07.648570 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d59f4a-3745-401a-a542-cd76b1e75e3f None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:07.681594 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 137/275] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:20:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:20:07.682448 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17eed5c-9d3c-4404-af37-9ad501b03c08 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:07.766588 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d59f4a-3745-401a-a542-cd76b1e75e3f None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:07.801568 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:07.804220 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb03db5-3c10-49c7-8646-5fb9214923a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:07.804932 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb03db5-3c10-49c7-8646-5fb9214923a1 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:07.805083 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17eed5c-9d3c-4404-af37-9ad501b03c08 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:07.805256 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb03db5-3c10-49c7-8646-5fb9214923a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:07.805490 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb03db5-3c10-49c7-8646-5fb9214923a1 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:07.806214 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb03db5-3c10-49c7-8646-5fb9214923a1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:07.806522 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb03db5-3c10-49c7-8646-5fb9214923a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:07.807932 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb03db5-3c10-49c7-8646-5fb9214923a1 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:07.821310 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 139/276] 10.222.0.26 () {56 vars in 1101 bytes} [Sat Aug 6 01:20:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:07.822083 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb03db5-3c10-49c7-8646-5fb9214923a1 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:07.838100 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:07.844451 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb43556-1aba-4f29-be9b-c9bc11f12dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:07.845125 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb43556-1aba-4f29-be9b-c9bc11f12dae None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:07.845410 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb43556-1aba-4f29-be9b-c9bc11f12dae None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:07.845683 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb43556-1aba-4f29-be9b-c9bc11f12dae None admin] PATH_INFO: `/v3/domains/default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:07.846870 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb43556-1aba-4f29-be9b-c9bc11f12dae None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:07.848305 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb43556-1aba-4f29-be9b-c9bc11f12dae None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:07.849936 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 138/277] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:20:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:07.850957 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb43556-1aba-4f29-be9b-c9bc11f12dae None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:07.934653 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb03db5-3c10-49c7-8646-5fb9214923a1 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:07.957248 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb43556-1aba-4f29-be9b-c9bc11f12dae None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:07.968434 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18435176-fb02-4a10-a2df-a646b4528255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:07.969101 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18435176-fb02-4a10-a2df-a646b4528255 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:07.969384 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18435176-fb02-4a10-a2df-a646b4528255 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:07.969657 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18435176-fb02-4a10-a2df-a646b4528255 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:07.970439 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18435176-fb02-4a10-a2df-a646b4528255 None admin] RBAC: Adding query filter params (role.id=6d8b63d350944d78812edbefb32ae8ee, scope.project.id=0bcfa6bfcd464fc9b0dc2079b24902bc, user.id=c1fcb82f07be4ffc85ac133727f3a453) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:07.970748 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18435176-fb02-4a10-a2df-a646b4528255 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:07.972176 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18435176-fb02-4a10-a2df-a646b4528255 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:07.989446 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 140/278] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Aug 6 01:20:07 2022] GET /identity/v3/role_assignments?user.id=c1fcb82f07be4ffc85ac133727f3a453&scope.project.id=0bcfa6bfcd464fc9b0dc2079b24902bc&role.id=6d8b63d350944d78812edbefb32ae8ee => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:07.989446 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18435176-fb02-4a10-a2df-a646b4528255 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:07.992672 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8bb191-9e6d-4446-8aa7-fa3b37d71533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:07.993391 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8bb191-9e6d-4446-8aa7-fa3b37d71533 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:07.993675 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8bb191-9e6d-4446-8aa7-fa3b37d71533 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:07.993953 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8bb191-9e6d-4446-8aa7-fa3b37d71533 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:08.006175 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8bb191-9e6d-4446-8aa7-fa3b37d71533 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:20:08.012034 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-0a8bb191-9e6d-4446-8aa7-fa3b37d71533 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 06 01:20:08.012034 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:08.012034 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:08.012034 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:08.012034 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:08.012034 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:08.012034 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:08.012034 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:08.012034 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:08.012034 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:08.012034 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:08.012034 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:08.012034 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:20:08.012034 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:20:08.012034 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:20:08.012034 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:20:08.013038 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:20:08.013038 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:20:08.013038 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:20:08.013038 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:20:08.013038 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:08.013038 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:08.013038 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:08.013038 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:08.013038 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:08.013038 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:08.013038 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:08.013038 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:08.013038 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:08.013038 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:08.013038 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:08.013038 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:08.014026 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:08.014026 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:08.014026 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:08.014026 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:08.014026 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:20:08.014026 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:20:08.014026 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:20:08.014026 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:20:08.014026 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:20:08.014026 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:20:08.014026 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 06 01:20:08.014026 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:20:08.014026 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 139/279] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Aug 6 01:20:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:20:08.014696 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8bb191-9e6d-4446-8aa7-fa3b37d71533 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:08.048358 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:08.082700 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:08.110020 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18435176-fb02-4a10-a2df-a646b4528255 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:08.131717 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8bb191-9e6d-4446-8aa7-fa3b37d71533 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:08.150454 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea39c32-d458-470f-b3d7-9fe0f685ccd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:08.151113 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea39c32-d458-470f-b3d7-9fe0f685ccd2 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:08.151401 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea39c32-d458-470f-b3d7-9fe0f685ccd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:08.152423 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea39c32-d458-470f-b3d7-9fe0f685ccd2 None admin] PATH_INFO: `/v3/users/2040b367cca449c3b645121fab91953d` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:08.168188 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea39c32-d458-470f-b3d7-9fe0f685ccd2 None admin] RBAC: Authorizing `identity:get_user(user_id=2040b367cca449c3b645121fab91953d)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:08.169792 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea39c32-d458-470f-b3d7-9fe0f685ccd2 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:08.171687 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 141/280] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Aug 6 01:20:07 2022] GET /identity/v3/users/2040b367cca449c3b645121fab91953d => generated 293 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:08.172695 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afac00e-43c2-4f4f-bc4b-81a06c2933ea None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:08.173033 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afac00e-43c2-4f4f-bc4b-81a06c2933ea None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:08.173423 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afac00e-43c2-4f4f-bc4b-81a06c2933ea None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:08.175430 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 142/281] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:20:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:20:08.176025 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afac00e-43c2-4f4f-bc4b-81a06c2933ea None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:08.181802 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e556f8f-f434-4982-9601-8bd667c59ff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:08.182443 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e556f8f-f434-4982-9601-8bd667c59ff7 None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:08.182729 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e556f8f-f434-4982-9601-8bd667c59ff7 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:08.183009 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e556f8f-f434-4982-9601-8bd667c59ff7 None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:08.183711 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e556f8f-f434-4982-9601-8bd667c59ff7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:08.185939 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e556f8f-f434-4982-9601-8bd667c59ff7 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:08.214118 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 140/282] 10.222.0.26 () {60 vars in 1132 bytes} [Sat Aug 6 01:20:08 2022] POST /identity/v3/users => generated 293 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:20:08.214972 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e556f8f-f434-4982-9601-8bd667c59ff7 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:08.295429 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afac00e-43c2-4f4f-bc4b-81a06c2933ea None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:08.313630 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:08.333278 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e556f8f-f434-4982-9601-8bd667c59ff7 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:08.338599 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718adf40-57cb-41a4-8cce-742413f39777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:08.339231 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718adf40-57cb-41a4-8cce-742413f39777 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:08.339684 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718adf40-57cb-41a4-8cce-742413f39777 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:08.339963 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718adf40-57cb-41a4-8cce-742413f39777 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:08.340824 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718adf40-57cb-41a4-8cce-742413f39777 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:08.341165 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718adf40-57cb-41a4-8cce-742413f39777 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:08.342611 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718adf40-57cb-41a4-8cce-742413f39777 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:08.344202 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:08.352793 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 143/283] 10.222.0.26 () {56 vars in 1101 bytes} [Sat Aug 6 01:20:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:08.353917 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eeed36-90b2-45f8-baad-bd244d7c8ff3 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:08.354200 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eeed36-90b2-45f8-baad-bd244d7c8ff3 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:08.354474 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eeed36-90b2-45f8-baad-bd244d7c8ff3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:08.373200 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-35eeed36-90b2-45f8-baad-bd244d7c8ff3 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:20:08.374417 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-35eeed36-90b2-45f8-baad-bd244d7c8ff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:20:08.380239 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea84611-4a81-486d-bfbc-4020ee39b74e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:08.380873 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea84611-4a81-486d-bfbc-4020ee39b74e None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:08.381184 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea84611-4a81-486d-bfbc-4020ee39b74e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:08.381462 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea84611-4a81-486d-bfbc-4020ee39b74e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:08.383014 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea84611-4a81-486d-bfbc-4020ee39b74e None admin] RBAC: Adding query filter params (role.id=61b3b93d37a047e4bb8369b46fe20db9, user.id=2040b367cca449c3b645121fab91953d, scope.system=all) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:08.383014 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea84611-4a81-486d-bfbc-4020ee39b74e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:08.384039 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea84611-4a81-486d-bfbc-4020ee39b74e None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:08.390597 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 141/284] 10.222.0.26 () {56 vars in 1308 bytes} [Sat Aug 6 01:20:08 2022] GET /identity/v3/role_assignments?user.id=2040b367cca449c3b645121fab91953d&scope.system=all&role.id=61b3b93d37a047e4bb8369b46fe20db9 => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:08.391903 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea84611-4a81-486d-bfbc-4020ee39b74e None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:08.479052 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 144/285] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:20:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:20:08.508174 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eeed36-90b2-45f8-baad-bd244d7c8ff3 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:08.539852 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea84611-4a81-486d-bfbc-4020ee39b74e None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:08.594221 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d5f5fb-a853-4324-82f4-acdd2fca4ae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:08.594958 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d5f5fb-a853-4324-82f4-acdd2fca4ae3 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:08.595252 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d5f5fb-a853-4324-82f4-acdd2fca4ae3 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:08.596341 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d5f5fb-a853-4324-82f4-acdd2fca4ae3 None admin] PATH_INFO: `/v3/users/ac3fe351cf5b49c395af5d46bc833105` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:08.597585 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d5f5fb-a853-4324-82f4-acdd2fca4ae3 None admin] RBAC: Authorizing `identity:get_user(user_id=ac3fe351cf5b49c395af5d46bc833105)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:08.599029 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d5f5fb-a853-4324-82f4-acdd2fca4ae3 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:08.602498 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 142/286] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Aug 6 01:20:08 2022] GET /identity/v3/users/ac3fe351cf5b49c395af5d46bc833105 => generated 297 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:08.608062 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d5f5fb-a853-4324-82f4-acdd2fca4ae3 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:08.627985 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:08.632852 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eeed36-90b2-45f8-baad-bd244d7c8ff3 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:08.667230 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:08.682757 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fe0497-d3b8-41a3-8302-7022fc6f6f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:08.683416 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fe0497-d3b8-41a3-8302-7022fc6f6f17 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:08.683698 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fe0497-d3b8-41a3-8302-7022fc6f6f17 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:08.683975 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fe0497-d3b8-41a3-8302-7022fc6f6f17 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:08.699712 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fe0497-d3b8-41a3-8302-7022fc6f6f17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:20:08.705705 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-22fe0497-d3b8-41a3-8302-7022fc6f6f17 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 06 01:20:08.705705 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:08.705705 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:08.705705 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:08.705705 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:08.705705 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:08.705705 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:08.705705 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:08.705705 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:08.705705 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:08.705705 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:08.705705 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:08.705705 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:20:08.705705 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:20:08.705705 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:20:08.705705 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:20:08.706605 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:20:08.706605 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:20:08.706605 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:20:08.706605 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:20:08.706605 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:08.706605 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:08.706605 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:08.706605 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:08.706605 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:08.706605 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:08.706605 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:08.706605 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:08.706605 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:08.706605 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:08.706605 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:08.706605 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:08.707810 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:08.707810 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:08.707810 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:08.707810 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:08.707810 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:20:08.707810 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:20:08.707810 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:20:08.707810 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:20:08.707810 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:20:08.707810 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:20:08.707810 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 06 01:20:08.707810 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:20:08.707810 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 145/287] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Aug 6 01:20:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:20:08.712619 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fe0497-d3b8-41a3-8302-7022fc6f6f17 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:08.740559 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d5f5fb-a853-4324-82f4-acdd2fca4ae3 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:08.792222 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92df6fcf-6b16-4441-929e-c496a25a0d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:08.793514 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92df6fcf-6b16-4441-929e-c496a25a0d1a None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:08.795528 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92df6fcf-6b16-4441-929e-c496a25a0d1a None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:08.796012 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92df6fcf-6b16-4441-929e-c496a25a0d1a None admin] PATH_INFO: `/v3/projects/0bcfa6bfcd464fc9b0dc2079b24902bc` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:08.797460 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92df6fcf-6b16-4441-929e-c496a25a0d1a None admin] RBAC: Authorizing `identity:get_project(project_id=0bcfa6bfcd464fc9b0dc2079b24902bc)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:08.799102 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92df6fcf-6b16-4441-929e-c496a25a0d1a None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:08.802389 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 143/288] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Aug 6 01:20:08 2022] GET /identity/v3/projects/0bcfa6bfcd464fc9b0dc2079b24902bc => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:20:08.808937 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92df6fcf-6b16-4441-929e-c496a25a0d1a None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:08.851743 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fe0497-d3b8-41a3-8302-7022fc6f6f17 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:08.916568 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a9bb84-ec5a-479c-9c65-fbecc7712ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:08.917251 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a9bb84-ec5a-479c-9c65-fbecc7712ba4 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:08.917538 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a9bb84-ec5a-479c-9c65-fbecc7712ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:08.917819 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a9bb84-ec5a-479c-9c65-fbecc7712ba4 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:08.918549 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a9bb84-ec5a-479c-9c65-fbecc7712ba4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:08.919403 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a9bb84-ec5a-479c-9c65-fbecc7712ba4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:08.920778 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a9bb84-ec5a-479c-9c65-fbecc7712ba4 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:08.944588 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 146/289] 10.222.0.26 () {56 vars in 1101 bytes} [Sat Aug 6 01:20:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:08.945253 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:08.958981 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a9bb84-ec5a-479c-9c65-fbecc7712ba4 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:08.965452 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92df6fcf-6b16-4441-929e-c496a25a0d1a None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:09.004178 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f536de6-1319-41aa-a619-f93c4592ecef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:09.004862 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f536de6-1319-41aa-a619-f93c4592ecef None admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:09.005442 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f536de6-1319-41aa-a619-f93c4592ecef None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:09.005727 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f536de6-1319-41aa-a619-f93c4592ecef None admin] PATH_INFO: `/v3/projects/0bcfa6bfcd464fc9b0dc2079b24902bc/users/ac3fe351cf5b49c395af5d46bc833105/roles/25efb2aed94e4d63b0b6cb8faf5a2ee7` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:09.009432 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f536de6-1319-41aa-a619-f93c4592ecef None admin] RBAC: Authorizing `identity:create_grant(project_id=0bcfa6bfcd464fc9b0dc2079b24902bc, user_id=ac3fe351cf5b49c395af5d46bc833105, role_id=25efb2aed94e4d63b0b6cb8faf5a2ee7)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:09.010891 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f536de6-1319-41aa-a619-f93c4592ecef None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:09.035460 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 144/290] 10.222.0.26 () {58 vars in 1439 bytes} [Sat Aug 6 01:20:08 2022] PUT /identity/v3/projects/0bcfa6bfcd464fc9b0dc2079b24902bc/users/ac3fe351cf5b49c395af5d46bc833105/roles/25efb2aed94e4d63b0b6cb8faf5a2ee7 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:20:09.076208 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:09.160387 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a9bb84-ec5a-479c-9c65-fbecc7712ba4 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:09.209889 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93446704-5555-4ef2-b668-9456b00b4b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:09.210782 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93446704-5555-4ef2-b668-9456b00b4b29 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:09.211232 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93446704-5555-4ef2-b668-9456b00b4b29 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:09.211651 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93446704-5555-4ef2-b668-9456b00b4b29 None admin] PATH_INFO: `/v3/users/16520a5cd4194361a52b3cf6d37a8053` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:09.212997 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93446704-5555-4ef2-b668-9456b00b4b29 None admin] RBAC: Authorizing `identity:get_user(user_id=16520a5cd4194361a52b3cf6d37a8053)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:09.214667 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93446704-5555-4ef2-b668-9456b00b4b29 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:09.217370 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 147/291] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Aug 6 01:20:08 2022] GET /identity/v3/users/16520a5cd4194361a52b3cf6d37a8053 => generated 297 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:20:09.223078 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f536de6-1319-41aa-a619-f93c4592ecef None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:09.331048 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f536de6-1319-41aa-a619-f93c4592ecef None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:09.368171 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557e50f3-cf80-4c52-aff9-434255d5431d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:09.369028 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557e50f3-cf80-4c52-aff9-434255d5431d None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:09.369478 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557e50f3-cf80-4c52-aff9-434255d5431d None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:09.369920 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557e50f3-cf80-4c52-aff9-434255d5431d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:09.391189 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:09.453409 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557e50f3-cf80-4c52-aff9-434255d5431d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:09.454877 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557e50f3-cf80-4c52-aff9-434255d5431d None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:09.570531 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 145/292] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Aug 6 01:20:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 01:20:09.576209 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93446704-5555-4ef2-b668-9456b00b4b29 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:09.699952 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93446704-5555-4ef2-b668-9456b00b4b29 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:09.716488 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:09.735732 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea92f939-e2d5-4035-a74a-f86b3972b0c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:09.736381 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea92f939-e2d5-4035-a74a-f86b3972b0c4 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:09.736663 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea92f939-e2d5-4035-a74a-f86b3972b0c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:09.736938 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea92f939-e2d5-4035-a74a-f86b3972b0c4 None admin] PATH_INFO: `/v3/projects/0bcfa6bfcd464fc9b0dc2079b24902bc` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:09.738231 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea92f939-e2d5-4035-a74a-f86b3972b0c4 None admin] RBAC: Authorizing `identity:get_project(project_id=0bcfa6bfcd464fc9b0dc2079b24902bc)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:09.739763 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea92f939-e2d5-4035-a74a-f86b3972b0c4 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:09.741578 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 148/293] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Aug 6 01:20:09 2022] GET /identity/v3/projects/0bcfa6bfcd464fc9b0dc2079b24902bc => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:09.749137 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557e50f3-cf80-4c52-aff9-434255d5431d None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:09.801236 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:09.860537 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557e50f3-cf80-4c52-aff9-434255d5431d None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:09.895974 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587066e5-baaa-4df7-bc32-10c8df616984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:09.896592 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587066e5-baaa-4df7-bc32-10c8df616984 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:09.896873 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587066e5-baaa-4df7-bc32-10c8df616984 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:09.897176 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587066e5-baaa-4df7-bc32-10c8df616984 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:09.897975 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587066e5-baaa-4df7-bc32-10c8df616984 None admin] RBAC: Adding query filter params (role.id=61b3b93d37a047e4bb8369b46fe20db9, scope.project.id=0bcfa6bfcd464fc9b0dc2079b24902bc, user.id=16520a5cd4194361a52b3cf6d37a8053) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:09.898285 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587066e5-baaa-4df7-bc32-10c8df616984 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:09.899717 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587066e5-baaa-4df7-bc32-10c8df616984 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:09.905966 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 146/294] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Aug 6 01:20:09 2022] GET /identity/v3/role_assignments?user.id=16520a5cd4194361a52b3cf6d37a8053&scope.project.id=0bcfa6bfcd464fc9b0dc2079b24902bc&role.id=61b3b93d37a047e4bb8369b46fe20db9 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:10.708801 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c71ecfd-f1e7-4b58-afed-a6d783b3975b None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:10.710125 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c71ecfd-f1e7-4b58-afed-a6d783b3975b None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:10.710609 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c71ecfd-f1e7-4b58-afed-a6d783b3975b None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:10.712388 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 149/295] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:20:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:20:10.718229 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e25ecc-0491-4e3d-b7e7-b1b8d9260c12 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:10.719033 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e25ecc-0491-4e3d-b7e7-b1b8d9260c12 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:10.719541 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e25ecc-0491-4e3d-b7e7-b1b8d9260c12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:10.739404 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-e6e25ecc-0491-4e3d-b7e7-b1b8d9260c12 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:20:10.740975 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-e6e25ecc-0491-4e3d-b7e7-b1b8d9260c12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:20:10.813794 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 147/296] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:20:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 01:20:10.821754 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f55541-021f-46c8-9ab8-898c86f918b1 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:10.822216 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f55541-021f-46c8-9ab8-898c86f918b1 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:10.822659 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f55541-021f-46c8-9ab8-898c86f918b1 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:10.824155 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 150/297] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:20:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:20:10.829225 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c202ad-eb95-4c2a-b071-7db4e07f8c65 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:10.829674 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c202ad-eb95-4c2a-b071-7db4e07f8c65 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:10.830110 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c202ad-eb95-4c2a-b071-7db4e07f8c65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:10.839558 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f55541-021f-46c8-9ab8-898c86f918b1 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:10.849951 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-d9c202ad-eb95-4c2a-b071-7db4e07f8c65 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:20:10.851520 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-d9c202ad-eb95-4c2a-b071-7db4e07f8c65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:20:10.885635 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:10.937329 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 148/298] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:20:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:20:10.953364 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c202ad-eb95-4c2a-b071-7db4e07f8c65 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:10.970771 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f55541-021f-46c8-9ab8-898c86f918b1 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:11.007140 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57799d3-3b2e-4e02-ab3c-7742155f7821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:11.007786 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57799d3-3b2e-4e02-ab3c-7742155f7821 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:11.008071 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57799d3-3b2e-4e02-ab3c-7742155f7821 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:11.008350 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57799d3-3b2e-4e02-ab3c-7742155f7821 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:11.023271 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57799d3-3b2e-4e02-ab3c-7742155f7821 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:20:11.030737 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-f57799d3-3b2e-4e02-ab3c-7742155f7821 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 06 01:20:11.030737 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:11.030737 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:11.030737 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:11.030737 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:11.030737 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:11.030737 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:11.030737 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:11.030737 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:11.030737 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:11.030737 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:11.030737 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:11.030737 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:20:11.030737 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:20:11.030737 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:20:11.030737 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:20:11.031640 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:20:11.031640 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:20:11.031640 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:20:11.031640 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:20:11.031640 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:11.031640 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:11.031640 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:11.031640 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:11.031640 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:11.031640 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:11.031640 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:11.031640 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:11.031640 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:11.031640 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:11.031640 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:11.031640 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:11.032420 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:11.032420 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:11.032420 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:11.032420 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:11.032420 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:20:11.032420 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:20:11.032420 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:20:11.032420 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:20:11.032420 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:20:11.032420 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:20:11.032420 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 06 01:20:11.032420 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:20:11.033011 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 151/299] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Aug 6 01:20:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:20:11.038394 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57799d3-3b2e-4e02-ab3c-7742155f7821 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:11.075319 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c202ad-eb95-4c2a-b071-7db4e07f8c65 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:11.089134 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:11.111879 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373e98c5-8780-4c87-837d-273ef44a5e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:11.112605 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373e98c5-8780-4c87-837d-273ef44a5e27 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:11.112894 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373e98c5-8780-4c87-837d-273ef44a5e27 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:11.113208 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373e98c5-8780-4c87-837d-273ef44a5e27 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:11.123978 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373e98c5-8780-4c87-837d-273ef44a5e27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:20:11.129134 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-373e98c5-8780-4c87-837d-273ef44a5e27 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 06 01:20:11.129134 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:11.129134 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:11.129134 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:11.129134 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:11.129134 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:11.129134 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:11.129134 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:11.129134 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:11.129134 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:11.129134 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:11.129134 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:11.129134 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:20:11.129134 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:20:11.129134 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:20:11.129134 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:20:11.129999 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:20:11.129999 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:20:11.129999 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:20:11.129999 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:20:11.129999 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:11.129999 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:11.129999 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:11.129999 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:11.129999 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:11.129999 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:11.129999 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:11.129999 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:11.129999 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:11.129999 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:11.129999 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:11.129999 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:11.130780 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:11.130780 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:11.130780 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:11.130780 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:11.130780 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:20:11.130780 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:20:11.130780 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:20:11.130780 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:20:11.130780 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:20:11.130780 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:20:11.130780 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 06 01:20:11.130780 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:20:11.130780 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 149/300] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Aug 6 01:20:10 2022] GET /identity/v3/roles/reader => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:20:11.133508 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4834504-d05a-4306-845b-15cc6e7cfc40 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:11.133796 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4834504-d05a-4306-845b-15cc6e7cfc40 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:11.134103 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4834504-d05a-4306-845b-15cc6e7cfc40 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:11.135675 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 150/301] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:20:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 01:20:11.136667 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4834504-d05a-4306-845b-15cc6e7cfc40 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:11.160509 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57799d3-3b2e-4e02-ab3c-7742155f7821 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:11.169361 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:11.195788 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92150ec-53b8-4402-a435-4716bc0b2ee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:11.196411 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92150ec-53b8-4402-a435-4716bc0b2ee8 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:11.196704 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92150ec-53b8-4402-a435-4716bc0b2ee8 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:11.196985 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92150ec-53b8-4402-a435-4716bc0b2ee8 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:11.197742 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92150ec-53b8-4402-a435-4716bc0b2ee8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:11.198054 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92150ec-53b8-4402-a435-4716bc0b2ee8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:11.199416 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92150ec-53b8-4402-a435-4716bc0b2ee8 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:11.208448 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 152/302] 10.222.0.26 () {56 vars in 1101 bytes} [Sat Aug 6 01:20:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:20:11.209562 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d363986-9974-4436-ad01-eadbd805231b None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:11.209859 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d363986-9974-4436-ad01-eadbd805231b None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:11.210158 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d363986-9974-4436-ad01-eadbd805231b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:11.228679 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-9d363986-9974-4436-ad01-eadbd805231b None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:20:11.229829 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-9d363986-9974-4436-ad01-eadbd805231b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:20:11.253551 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4834504-d05a-4306-845b-15cc6e7cfc40 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:11.293491 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d718064-65e6-4eac-96c3-79b84a2ccf31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:11.294679 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d718064-65e6-4eac-96c3-79b84a2ccf31 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:11.294960 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d718064-65e6-4eac-96c3-79b84a2ccf31 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:11.295248 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d718064-65e6-4eac-96c3-79b84a2ccf31 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:11.295975 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d718064-65e6-4eac-96c3-79b84a2ccf31 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:11.296287 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d718064-65e6-4eac-96c3-79b84a2ccf31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:11.297693 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d718064-65e6-4eac-96c3-79b84a2ccf31 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:11.301095 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 153/303] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:20:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:20:11.302098 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d363986-9974-4436-ad01-eadbd805231b None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:11.309096 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 151/304] 10.222.0.26 () {56 vars in 1101 bytes} [Sat Aug 6 01:20:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:11.314897 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d718064-65e6-4eac-96c3-79b84a2ccf31 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:11.351366 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:11.416436 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:11.426888 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d718064-65e6-4eac-96c3-79b84a2ccf31 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:11.427706 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d363986-9974-4436-ad01-eadbd805231b None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:11.461017 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56b8d3f-33f8-4314-9989-490edbe77107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:11.461207 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c327a00-6c29-4eb3-ac96-0efacdfefd08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:11.461690 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56b8d3f-33f8-4314-9989-490edbe77107 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:11.461795 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c327a00-6c29-4eb3-ac96-0efacdfefd08 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:11.461979 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56b8d3f-33f8-4314-9989-490edbe77107 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:11.462081 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c327a00-6c29-4eb3-ac96-0efacdfefd08 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:11.462249 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c327a00-6c29-4eb3-ac96-0efacdfefd08 None admin] PATH_INFO: `/v3/users/2040b367cca449c3b645121fab91953d` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:11.462356 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56b8d3f-33f8-4314-9989-490edbe77107 None admin] PATH_INFO: `/v3/users/acd8321448b64ce9b1d05e6455cec4c4` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:11.463449 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c327a00-6c29-4eb3-ac96-0efacdfefd08 None admin] RBAC: Authorizing `identity:get_user(user_id=2040b367cca449c3b645121fab91953d)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:11.464873 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c327a00-6c29-4eb3-ac96-0efacdfefd08 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:11.467068 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 154/305] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Aug 6 01:20:11 2022] GET /identity/v3/users/2040b367cca449c3b645121fab91953d => generated 293 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:11.467820 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c327a00-6c29-4eb3-ac96-0efacdfefd08 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:11.477919 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56b8d3f-33f8-4314-9989-490edbe77107 None admin] RBAC: Authorizing `identity:get_user(user_id=acd8321448b64ce9b1d05e6455cec4c4)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:11.479417 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56b8d3f-33f8-4314-9989-490edbe77107 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:11.481213 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 152/306] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Aug 6 01:20:11 2022] GET /identity/v3/users/acd8321448b64ce9b1d05e6455cec4c4 => generated 293 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:11.482303 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cc0156-c682-4825-a1e8-b7a0275dc04f None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:11.482569 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cc0156-c682-4825-a1e8-b7a0275dc04f None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:11.482849 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cc0156-c682-4825-a1e8-b7a0275dc04f None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:11.484065 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 153/307] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:20:11 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:20:11.484747 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cc0156-c682-4825-a1e8-b7a0275dc04f None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:11.570284 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c327a00-6c29-4eb3-ac96-0efacdfefd08 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:11.572558 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:11.585250 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cc0156-c682-4825-a1e8-b7a0275dc04f None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:11.606380 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797917d2-a816-4ec5-9b6f-a834901fd2ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:11.607110 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797917d2-a816-4ec5-9b6f-a834901fd2ed None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:11.607393 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797917d2-a816-4ec5-9b6f-a834901fd2ed None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:11.607663 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797917d2-a816-4ec5-9b6f-a834901fd2ed None admin] PATH_INFO: `/v3/domains/default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:11.608818 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797917d2-a816-4ec5-9b6f-a834901fd2ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:11.610305 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797917d2-a816-4ec5-9b6f-a834901fd2ed None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:11.612045 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 155/308] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:20:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:11.612623 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797917d2-a816-4ec5-9b6f-a834901fd2ed None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:11.623098 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2098ea14-a3cd-488f-a4f6-df570af733c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:11.623766 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2098ea14-a3cd-488f-a4f6-df570af733c7 None admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:11.624049 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2098ea14-a3cd-488f-a4f6-df570af733c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:11.624328 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2098ea14-a3cd-488f-a4f6-df570af733c7 None admin] PATH_INFO: `/v3/system/users/2040b367cca449c3b645121fab91953d/roles/61b3b93d37a047e4bb8369b46fe20db9` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:11.625827 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2098ea14-a3cd-488f-a4f6-df570af733c7 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=2040b367cca449c3b645121fab91953d, role_id=61b3b93d37a047e4bb8369b46fe20db9)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:11.627194 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2098ea14-a3cd-488f-a4f6-df570af733c7 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:11.636048 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 154/309] 10.222.0.26 () {58 vars in 1334 bytes} [Sat Aug 6 01:20:11 2022] PUT /identity/v3/system/users/2040b367cca449c3b645121fab91953d/roles/61b3b93d37a047e4bb8369b46fe20db9 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:20:11.649262 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943f01d8-672b-4d0b-9099-36151b53b2cf None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:11.649262 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943f01d8-672b-4d0b-9099-36151b53b2cf None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:11.649262 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943f01d8-672b-4d0b-9099-36151b53b2cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:11.658116 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-943f01d8-672b-4d0b-9099-36151b53b2cf None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:20:11.659365 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-943f01d8-672b-4d0b-9099-36151b53b2cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:20:11.668206 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:11.730436 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797917d2-a816-4ec5-9b6f-a834901fd2ed None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:11.738862 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 155/310] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:20:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:20:11.739711 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943f01d8-672b-4d0b-9099-36151b53b2cf None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:11.770429 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9a9ab2-26b7-477e-b6d1-0995faf71750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:11.771417 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9a9ab2-26b7-477e-b6d1-0995faf71750 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:11.771919 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9a9ab2-26b7-477e-b6d1-0995faf71750 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:11.772388 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9a9ab2-26b7-477e-b6d1-0995faf71750 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:11.773586 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9a9ab2-26b7-477e-b6d1-0995faf71750 None admin] RBAC: Adding query filter params (role.id=25efb2aed94e4d63b0b6cb8faf5a2ee7, user.id=acd8321448b64ce9b1d05e6455cec4c4, scope.system=all) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:11.774130 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9a9ab2-26b7-477e-b6d1-0995faf71750 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:11.775798 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9a9ab2-26b7-477e-b6d1-0995faf71750 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:11.782447 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 156/311] 10.222.0.26 () {56 vars in 1308 bytes} [Sat Aug 6 01:20:11 2022] GET /identity/v3/role_assignments?user.id=acd8321448b64ce9b1d05e6455cec4c4&scope.system=all&role.id=25efb2aed94e4d63b0b6cb8faf5a2ee7 => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:11.783844 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9a9ab2-26b7-477e-b6d1-0995faf71750 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:11.828051 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:11.900177 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943f01d8-672b-4d0b-9099-36151b53b2cf None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:11.903110 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9a9ab2-26b7-477e-b6d1-0995faf71750 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:11.945430 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f02f45-9d8a-43f9-a4f9-baac8fa311ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:11.946056 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f02f45-9d8a-43f9-a4f9-baac8fa311ca None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:11.947059 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f02f45-9d8a-43f9-a4f9-baac8fa311ca None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:11.947349 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f02f45-9d8a-43f9-a4f9-baac8fa311ca None admin] PATH_INFO: `/v3/groups` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:11.948023 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f02f45-9d8a-43f9-a4f9-baac8fa311ca None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:11.948225 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c430c92-a955-4673-8546-9dc59a727684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:11.949117 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c430c92-a955-4673-8546-9dc59a727684 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:11.949518 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f02f45-9d8a-43f9-a4f9-baac8fa311ca None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:11.949801 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c430c92-a955-4673-8546-9dc59a727684 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:11.951187 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c430c92-a955-4673-8546-9dc59a727684 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:11.963929 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c430c92-a955-4673-8546-9dc59a727684 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:20:11.970293 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 156/312] 10.222.0.26 () {60 vars in 1134 bytes} [Sat Aug 6 01:20:11 2022] POST /identity/v3/groups => generated 229 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:20:11.971448 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-5c430c92-a955-4673-8546-9dc59a727684 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 06 01:20:11.971448 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:11.971448 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:11.971448 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:11.971448 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:11.971448 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:11.971448 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:11.971448 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:11.971448 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:11.971448 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:11.971448 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:11.971448 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:11.971448 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:20:11.971448 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:20:11.971448 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:20:11.971448 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:20:11.972296 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:20:11.972296 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:20:11.972296 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:20:11.972296 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:20:11.972296 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:11.972296 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:11.972296 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:11.972296 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:11.972296 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:11.972296 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:11.972296 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:11.972296 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:11.972296 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:11.972296 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:11.972296 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:11.972296 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:11.973142 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:11.973142 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:11.973142 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:11.973142 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:11.973142 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:20:11.973142 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:20:11.973142 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:20:11.973142 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:20:11.973142 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:20:11.973142 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:20:11.973142 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 06 01:20:11.973142 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:20:11.975398 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 157/313] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Aug 6 01:20:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:20:11.983291 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f02f45-9d8a-43f9-a4f9-baac8fa311ca None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:11.990975 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:12.110139 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f02f45-9d8a-43f9-a4f9-baac8fa311ca None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:12.146961 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff88e2fd-8c4c-4a9c-a8cf-d974f4d970ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:12.147653 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff88e2fd-8c4c-4a9c-a8cf-d974f4d970ca None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:12.147934 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff88e2fd-8c4c-4a9c-a8cf-d974f4d970ca None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:12.148211 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff88e2fd-8c4c-4a9c-a8cf-d974f4d970ca None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:12.148934 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff88e2fd-8c4c-4a9c-a8cf-d974f4d970ca None admin] RBAC: Adding query filter params (name=reader) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:12.149268 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff88e2fd-8c4c-4a9c-a8cf-d974f4d970ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:12.150637 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff88e2fd-8c4c-4a9c-a8cf-d974f4d970ca None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:12.159631 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 157/314] 10.222.0.26 () {56 vars in 1101 bytes} [Sat Aug 6 01:20:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:12.164850 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c430c92-a955-4673-8546-9dc59a727684 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:12.267856 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:12.275460 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c430c92-a955-4673-8546-9dc59a727684 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:12.311305 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8db99f8-cc07-40f8-971a-7e9faf7be2c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:12.311997 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8db99f8-cc07-40f8-971a-7e9faf7be2c8 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:12.312280 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8db99f8-cc07-40f8-971a-7e9faf7be2c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:12.312654 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8db99f8-cc07-40f8-971a-7e9faf7be2c8 None admin] PATH_INFO: `/v3/users/ac3fe351cf5b49c395af5d46bc833105` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:12.313871 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8db99f8-cc07-40f8-971a-7e9faf7be2c8 None admin] RBAC: Authorizing `identity:get_user(user_id=ac3fe351cf5b49c395af5d46bc833105)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:12.315307 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8db99f8-cc07-40f8-971a-7e9faf7be2c8 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:12.317266 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 158/315] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Aug 6 01:20:12 2022] GET /identity/v3/users/ac3fe351cf5b49c395af5d46bc833105 => generated 297 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:20:12.322428 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff88e2fd-8c4c-4a9c-a8cf-d974f4d970ca None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:12.389421 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:12.427694 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff88e2fd-8c4c-4a9c-a8cf-d974f4d970ca None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:12.463227 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15dab81-283f-4e02-b617-c9089d2c855f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:12.463855 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15dab81-283f-4e02-b617-c9089d2c855f None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:12.464135 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15dab81-283f-4e02-b617-c9089d2c855f None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:12.464410 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15dab81-283f-4e02-b617-c9089d2c855f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:12.535017 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15dab81-283f-4e02-b617-c9089d2c855f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:12.536394 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15dab81-283f-4e02-b617-c9089d2c855f None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:12.598031 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:12.637024 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 158/316] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Aug 6 01:20:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 01:20:12.642922 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8db99f8-cc07-40f8-971a-7e9faf7be2c8 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:12.750766 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8db99f8-cc07-40f8-971a-7e9faf7be2c8 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:12.788654 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b266a6fb-0d5b-4bcd-a2f1-a37cdd1eea94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:12.789279 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b266a6fb-0d5b-4bcd-a2f1-a37cdd1eea94 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:12.789572 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b266a6fb-0d5b-4bcd-a2f1-a37cdd1eea94 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:12.789848 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b266a6fb-0d5b-4bcd-a2f1-a37cdd1eea94 None admin] PATH_INFO: `/v3/projects/0bcfa6bfcd464fc9b0dc2079b24902bc` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:12.791106 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b266a6fb-0d5b-4bcd-a2f1-a37cdd1eea94 None admin] RBAC: Authorizing `identity:get_project(project_id=0bcfa6bfcd464fc9b0dc2079b24902bc)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:12.792595 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b266a6fb-0d5b-4bcd-a2f1-a37cdd1eea94 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:12.795156 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 159/317] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Aug 6 01:20:12 2022] GET /identity/v3/projects/0bcfa6bfcd464fc9b0dc2079b24902bc => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:12.806265 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15dab81-283f-4e02-b617-c9089d2c855f None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:12.912901 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15dab81-283f-4e02-b617-c9089d2c855f None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:12.949251 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf85163-4be7-4547-857b-ab13945d71ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:12.949922 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf85163-4be7-4547-857b-ab13945d71ef None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:12.950203 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf85163-4be7-4547-857b-ab13945d71ef None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:12.950486 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf85163-4be7-4547-857b-ab13945d71ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:12.951273 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf85163-4be7-4547-857b-ab13945d71ef None admin] RBAC: Adding query filter params (role.id=25efb2aed94e4d63b0b6cb8faf5a2ee7, scope.project.id=0bcfa6bfcd464fc9b0dc2079b24902bc, user.id=ac3fe351cf5b49c395af5d46bc833105) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:12.951584 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf85163-4be7-4547-857b-ab13945d71ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:12.953006 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf85163-4be7-4547-857b-ab13945d71ef None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:12.959785 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 159/318] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Aug 6 01:20:12 2022] GET /identity/v3/role_assignments?user.id=ac3fe351cf5b49c395af5d46bc833105&scope.project.id=0bcfa6bfcd464fc9b0dc2079b24902bc&role.id=25efb2aed94e4d63b0b6cb8faf5a2ee7 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:13.704879 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0e3dc1-1246-47e6-b7e8-5bc4d4b9b016 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:13.705275 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0e3dc1-1246-47e6-b7e8-5bc4d4b9b016 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:13.705465 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0e3dc1-1246-47e6-b7e8-5bc4d4b9b016 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:13.709088 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 160/319] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:20:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:20:13.712057 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ede800-6d57-4eda-beb6-f3b8221861cf None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:13.712485 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ede800-6d57-4eda-beb6-f3b8221861cf None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:13.712769 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ede800-6d57-4eda-beb6-f3b8221861cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:13.732608 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-96ede800-6d57-4eda-beb6-f3b8221861cf None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:20:13.733890 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-96ede800-6d57-4eda-beb6-f3b8221861cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:20:13.758776 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:13.806305 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 160/320] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:20:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:20:13.829284 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0e3dc1-1246-47e6-b7e8-5bc4d4b9b016 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:13.862019 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:13.871328 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3de28d-af0f-4e6e-bf29-bce56cb2cc48 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:13.871624 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3de28d-af0f-4e6e-bf29-bce56cb2cc48 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:13.871971 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3de28d-af0f-4e6e-bf29-bce56cb2cc48 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:13.873408 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 161/321] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:20:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 01:20:13.878479 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f3e921-1ccb-459e-89dc-61e546dcda37 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:13.878766 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f3e921-1ccb-459e-89dc-61e546dcda37 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:13.879047 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f3e921-1ccb-459e-89dc-61e546dcda37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:13.897482 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-04f3e921-1ccb-459e-89dc-61e546dcda37 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:20:13.898666 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-04f3e921-1ccb-459e-89dc-61e546dcda37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:20:13.938519 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0e3dc1-1246-47e6-b7e8-5bc4d4b9b016 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:13.969967 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 162/322] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:20:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 01:20:13.980381 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2160deae-9571-4099-9f8a-d4fec4240994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:13.981102 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2160deae-9571-4099-9f8a-d4fec4240994 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:13.981892 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2160deae-9571-4099-9f8a-d4fec4240994 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:13.982165 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2160deae-9571-4099-9f8a-d4fec4240994 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:13.992694 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2160deae-9571-4099-9f8a-d4fec4240994 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:20:13.993963 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f3e921-1ccb-459e-89dc-61e546dcda37 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:13.997806 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-2160deae-9571-4099-9f8a-d4fec4240994 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 06 01:20:13.997806 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:13.997806 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:13.997806 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:13.997806 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:13.997806 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:13.997806 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:13.997806 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:13.997806 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:13.997806 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:13.997806 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:13.997806 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:13.997806 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:20:13.997806 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:20:13.997806 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:20:13.997806 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:20:13.998873 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:20:13.998873 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:20:13.998873 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:20:13.998873 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:20:13.998873 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:13.998873 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:13.998873 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:13.998873 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:13.998873 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:13.998873 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:13.998873 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:13.998873 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:13.998873 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:13.998873 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:13.998873 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:13.998873 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:13.999659 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:13.999659 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:13.999659 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:13.999659 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:13.999659 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:20:13.999659 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:20:13.999659 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:20:13.999659 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:20:13.999659 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:20:13.999659 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:20:13.999659 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 06 01:20:13.999659 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:20:13.999659 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 161/323] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Aug 6 01:20:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:20:14.006474 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2160deae-9571-4099-9f8a-d4fec4240994 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:14.091782 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:14.106722 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f3e921-1ccb-459e-89dc-61e546dcda37 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:14.109514 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:14.121098 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2160deae-9571-4099-9f8a-d4fec4240994 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:14.142216 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af301c4-f74b-4d20-8fd5-c4102a75dc44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:14.142864 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af301c4-f74b-4d20-8fd5-c4102a75dc44 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:14.143146 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af301c4-f74b-4d20-8fd5-c4102a75dc44 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:14.143421 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af301c4-f74b-4d20-8fd5-c4102a75dc44 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:14.154410 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af301c4-f74b-4d20-8fd5-c4102a75dc44 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:20:14.158954 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f28ebe6-5628-496c-9dea-7db850382b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:14.159336 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-2af301c4-f74b-4d20-8fd5-c4102a75dc44 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 06 01:20:14.159336 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:14.159336 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:14.159336 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:14.159336 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:14.159336 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:14.159336 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:14.159336 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:14.159336 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:14.159336 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:14.159336 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:14.159336 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:14.159336 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:20:14.159336 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:20:14.159336 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:20:14.159336 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:20:14.160274 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:20:14.160274 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:20:14.160274 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:20:14.160274 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:20:14.160274 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:14.160274 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:14.160274 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:14.160274 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:14.160274 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:14.160274 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:14.160274 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:14.160274 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:14.160274 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:14.160274 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:14.160274 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:14.160274 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:14.161079 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:14.161079 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:14.161079 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:14.161079 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:14.161079 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:20:14.161079 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:20:14.161079 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:20:14.161079 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:20:14.161079 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:20:14.161079 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:20:14.161079 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 06 01:20:14.161079 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:20:14.161679 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f28ebe6-5628-496c-9dea-7db850382b0c None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:14.161679 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f28ebe6-5628-496c-9dea-7db850382b0c None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:14.161679 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f28ebe6-5628-496c-9dea-7db850382b0c None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:14.161679 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f28ebe6-5628-496c-9dea-7db850382b0c None admin] RBAC: Adding query filter params (name=member) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:14.161679 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f28ebe6-5628-496c-9dea-7db850382b0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:14.162469 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f28ebe6-5628-496c-9dea-7db850382b0c None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:14.165318 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 163/324] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Aug 6 01:20:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:20:14.166572 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff82bd2b-3ffa-455a-9c79-e300720ba5ff None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:14.166866 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff82bd2b-3ffa-455a-9c79-e300720ba5ff None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:14.167153 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff82bd2b-3ffa-455a-9c79-e300720ba5ff None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:14.168425 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 164/325] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:20:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:20:14.169231 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff82bd2b-3ffa-455a-9c79-e300720ba5ff None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:14.175098 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 162/326] 10.222.0.26 () {56 vars in 1101 bytes} [Sat Aug 6 01:20:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:14.175098 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e116f620-d720-46ec-987e-a6bcc0e523b0 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:14.175098 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e116f620-d720-46ec-987e-a6bcc0e523b0 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:14.175098 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e116f620-d720-46ec-987e-a6bcc0e523b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:14.192615 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-e116f620-d720-46ec-987e-a6bcc0e523b0 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:20:14.193858 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-e116f620-d720-46ec-987e-a6bcc0e523b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:20:14.259613 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 163/327] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:20:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 01:20:14.260361 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e116f620-d720-46ec-987e-a6bcc0e523b0 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:14.272556 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff82bd2b-3ffa-455a-9c79-e300720ba5ff None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:14.307482 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c39176-3984-4eab-add2-2bfeb3737c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:14.308193 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c39176-3984-4eab-add2-2bfeb3737c5d None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:14.308475 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c39176-3984-4eab-add2-2bfeb3737c5d None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:14.308751 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c39176-3984-4eab-add2-2bfeb3737c5d None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:14.309853 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c39176-3984-4eab-add2-2bfeb3737c5d None admin] RBAC: Adding query filter params (name=reader) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:14.310181 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c39176-3984-4eab-add2-2bfeb3737c5d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:14.311542 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c39176-3984-4eab-add2-2bfeb3737c5d None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:14.319564 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 165/328] 10.222.0.26 () {56 vars in 1101 bytes} [Sat Aug 6 01:20:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:14.320354 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c39176-3984-4eab-add2-2bfeb3737c5d None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:14.334294 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:14.345272 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:14.361169 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e116f620-d720-46ec-987e-a6bcc0e523b0 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:14.396655 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65cd2ba-72a8-4334-a1dd-2104592a9665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:14.397327 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65cd2ba-72a8-4334-a1dd-2104592a9665 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:14.397606 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65cd2ba-72a8-4334-a1dd-2104592a9665 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:14.397882 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65cd2ba-72a8-4334-a1dd-2104592a9665 None admin] PATH_INFO: `/v3/users/2040b367cca449c3b645121fab91953d` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:14.399051 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65cd2ba-72a8-4334-a1dd-2104592a9665 None admin] RBAC: Authorizing `identity:get_user(user_id=2040b367cca449c3b645121fab91953d)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:14.400485 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65cd2ba-72a8-4334-a1dd-2104592a9665 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:14.404536 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 164/329] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Aug 6 01:20:14 2022] GET /identity/v3/users/2040b367cca449c3b645121fab91953d => generated 293 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:14.404536 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65cd2ba-72a8-4334-a1dd-2104592a9665 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:14.426251 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c39176-3984-4eab-add2-2bfeb3737c5d None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:14.460021 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b878fc-7038-4c41-8e8d-5f3c1fb7c596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:14.460641 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b878fc-7038-4c41-8e8d-5f3c1fb7c596 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:14.460918 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b878fc-7038-4c41-8e8d-5f3c1fb7c596 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:14.461222 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b878fc-7038-4c41-8e8d-5f3c1fb7c596 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:14.462368 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b878fc-7038-4c41-8e8d-5f3c1fb7c596 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:14.463785 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b878fc-7038-4c41-8e8d-5f3c1fb7c596 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:14.468225 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 166/330] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:20:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:14.468225 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b878fc-7038-4c41-8e8d-5f3c1fb7c596 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:14.506747 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65cd2ba-72a8-4334-a1dd-2104592a9665 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:14.539881 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b3bbeb-d8e5-4c05-9a2f-bda1cdd96fa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:14.540442 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b3bbeb-d8e5-4c05-9a2f-bda1cdd96fa4 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:14.540717 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b3bbeb-d8e5-4c05-9a2f-bda1cdd96fa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:14.540988 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b3bbeb-d8e5-4c05-9a2f-bda1cdd96fa4 None admin] PATH_INFO: `/v3/users/acd8321448b64ce9b1d05e6455cec4c4` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:14.542175 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b3bbeb-d8e5-4c05-9a2f-bda1cdd96fa4 None admin] RBAC: Authorizing `identity:get_user(user_id=acd8321448b64ce9b1d05e6455cec4c4)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:14.543729 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b3bbeb-d8e5-4c05-9a2f-bda1cdd96fa4 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:14.545704 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 165/331] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Aug 6 01:20:14 2022] GET /identity/v3/users/acd8321448b64ce9b1d05e6455cec4c4 => generated 293 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:14.546437 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b3bbeb-d8e5-4c05-9a2f-bda1cdd96fa4 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:14.546897 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:14.568762 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:14.569841 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b878fc-7038-4c41-8e8d-5f3c1fb7c596 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:14.604195 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59606c33-12c4-4923-946b-02a81b7c19bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:14.604811 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59606c33-12c4-4923-946b-02a81b7c19bf None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:14.605132 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59606c33-12c4-4923-946b-02a81b7c19bf None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:14.605417 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59606c33-12c4-4923-946b-02a81b7c19bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:14.606273 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59606c33-12c4-4923-946b-02a81b7c19bf None admin] RBAC: Adding query filter params (role.id=61b3b93d37a047e4bb8369b46fe20db9, user.id=2040b367cca449c3b645121fab91953d, scope.system=all) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:14.606585 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59606c33-12c4-4923-946b-02a81b7c19bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:14.608020 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59606c33-12c4-4923-946b-02a81b7c19bf None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:14.619369 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 167/332] 10.222.0.26 () {56 vars in 1308 bytes} [Sat Aug 6 01:20:14 2022] GET /identity/v3/role_assignments?user.id=2040b367cca449c3b645121fab91953d&scope.system=all&role.id=61b3b93d37a047e4bb8369b46fe20db9 => generated 510 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:14.620136 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59606c33-12c4-4923-946b-02a81b7c19bf None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:14.651700 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b3bbeb-d8e5-4c05-9a2f-bda1cdd96fa4 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:14.691465 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e47f592-93bf-45f7-a8bb-6af1a1c02747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:14.692108 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e47f592-93bf-45f7-a8bb-6af1a1c02747 None admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:14.692397 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e47f592-93bf-45f7-a8bb-6af1a1c02747 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:14.692662 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e47f592-93bf-45f7-a8bb-6af1a1c02747 None admin] PATH_INFO: `/v3/groups` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:14.693380 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e47f592-93bf-45f7-a8bb-6af1a1c02747 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:14.694852 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e47f592-93bf-45f7-a8bb-6af1a1c02747 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:14.717145 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 166/333] 10.222.0.26 () {60 vars in 1134 bytes} [Sat Aug 6 01:20:14 2022] POST /identity/v3/groups => generated 226 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:20:14.762792 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59606c33-12c4-4923-946b-02a81b7c19bf None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:14.810124 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd8839f-6741-463f-af78-5821bd9b4ac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:14.811557 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd8839f-6741-463f-af78-5821bd9b4ac2 None admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:14.811845 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd8839f-6741-463f-af78-5821bd9b4ac2 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:14.812128 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd8839f-6741-463f-af78-5821bd9b4ac2 None admin] PATH_INFO: `/v3/system/users/acd8321448b64ce9b1d05e6455cec4c4/roles/25efb2aed94e4d63b0b6cb8faf5a2ee7` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:14.813748 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd8839f-6741-463f-af78-5821bd9b4ac2 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=acd8321448b64ce9b1d05e6455cec4c4, role_id=25efb2aed94e4d63b0b6cb8faf5a2ee7)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:14.815213 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd8839f-6741-463f-af78-5821bd9b4ac2 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:14.817913 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:14.824915 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 168/334] 10.222.0.26 () {58 vars in 1334 bytes} [Sat Aug 6 01:20:14 2022] PUT /identity/v3/system/users/acd8321448b64ce9b1d05e6455cec4c4/roles/25efb2aed94e4d63b0b6cb8faf5a2ee7 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:20:17.130724 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d98981-b223-4320-bb9c-ea5385745f8d None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:17.131233 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d98981-b223-4320-bb9c-ea5385745f8d None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:17.131321 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d98981-b223-4320-bb9c-ea5385745f8d None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:17.132883 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 167/335] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:20:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:20:17.140255 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa7036c-7358-45e9-a860-492bc5b6405f None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:17.140772 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa7036c-7358-45e9-a860-492bc5b6405f None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:17.140850 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa7036c-7358-45e9-a860-492bc5b6405f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:17.165711 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-1fa7036c-7358-45e9-a860-492bc5b6405f None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:20:17.166906 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-1fa7036c-7358-45e9-a860-492bc5b6405f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:20:17.251974 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 169/336] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:20:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:20:17.286153 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d98981-b223-4320-bb9c-ea5385745f8d None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:17.389487 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:17.422123 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d98981-b223-4320-bb9c-ea5385745f8d None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:17.425264 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbc3ef0-8aa8-4d33-abd5-28c7b14412e6 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:17.425791 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbc3ef0-8aa8-4d33-abd5-28c7b14412e6 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:17.426286 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbc3ef0-8aa8-4d33-abd5-28c7b14412e6 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:17.427847 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 170/337] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:20:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:20:17.433232 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9871a0f8-283c-4899-8c58-0edcfe15eaf5 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:17.433810 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9871a0f8-283c-4899-8c58-0edcfe15eaf5 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:17.434906 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9871a0f8-283c-4899-8c58-0edcfe15eaf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:17.467482 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-9871a0f8-283c-4899-8c58-0edcfe15eaf5 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:20:17.468699 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-9871a0f8-283c-4899-8c58-0edcfe15eaf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:20:17.473236 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570242ad-323e-4186-a0ef-77c1a3286fdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:17.473973 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570242ad-323e-4186-a0ef-77c1a3286fdf None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:17.474258 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570242ad-323e-4186-a0ef-77c1a3286fdf None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:17.474528 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570242ad-323e-4186-a0ef-77c1a3286fdf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:17.497799 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570242ad-323e-4186-a0ef-77c1a3286fdf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:20:17.504619 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-570242ad-323e-4186-a0ef-77c1a3286fdf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 01:20:17.504619 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:17.504619 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:17.504619 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:17.504619 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:17.504619 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:17.504619 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:17.504619 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:17.504619 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:17.504619 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:17.504619 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:17.504619 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:17.504619 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:20:17.504619 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:20:17.504619 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:20:17.504619 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:20:17.508083 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:20:17.508083 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:20:17.508083 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:20:17.508083 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:20:17.508083 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:17.508083 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:17.508083 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:17.508083 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:17.508083 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:17.508083 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:17.508083 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:17.508083 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:17.508083 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:17.508083 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:17.508083 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:17.508083 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:17.508861 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:17.508861 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:17.508861 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:17.508861 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:17.508861 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:20:17.508861 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:20:17.508861 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:20:17.508861 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:20:17.508861 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:20:17.508861 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:20:17.508861 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 01:20:17.508861 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:20:17.511513 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 168/338] 10.222.0.26 () {56 vars in 1114 bytes} [Sat Aug 6 01:20:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:20:17.517861 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570242ad-323e-4186-a0ef-77c1a3286fdf None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:17.611052 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 171/339] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:20:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:20:17.640765 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570242ad-323e-4186-a0ef-77c1a3286fdf None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:17.647899 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9871a0f8-283c-4899-8c58-0edcfe15eaf5 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:17.692244 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d2612b-f178-4bd7-8417-cf34ffddb35e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:17.693180 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d2612b-f178-4bd7-8417-cf34ffddb35e None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:17.693607 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d2612b-f178-4bd7-8417-cf34ffddb35e None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:17.694054 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d2612b-f178-4bd7-8417-cf34ffddb35e None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:17.694913 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d2612b-f178-4bd7-8417-cf34ffddb35e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:17.695380 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d2612b-f178-4bd7-8417-cf34ffddb35e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:17.696910 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d2612b-f178-4bd7-8417-cf34ffddb35e None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:17.719773 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 169/340] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Aug 6 01:20:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:20:17.721448 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d65df2-1ffb-4f22-8ced-c7522c2d0946 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:17.721872 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d65df2-1ffb-4f22-8ced-c7522c2d0946 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:17.722282 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d65df2-1ffb-4f22-8ced-c7522c2d0946 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:17.723718 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 170/341] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:20:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:20:17.729327 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85a4c41-3dfa-4b4d-916e-632b63fdafce None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:17.729775 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85a4c41-3dfa-4b4d-916e-632b63fdafce None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:17.730191 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85a4c41-3dfa-4b4d-916e-632b63fdafce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:17.741777 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:17.758374 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-c85a4c41-3dfa-4b4d-916e-632b63fdafce None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:20:17.759802 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-c85a4c41-3dfa-4b4d-916e-632b63fdafce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:20:17.762785 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:17.820640 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9871a0f8-283c-4899-8c58-0edcfe15eaf5 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:17.854819 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 171/342] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:20:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 01:20:17.856667 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85a4c41-3dfa-4b4d-916e-632b63fdafce None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:17.862080 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602c3e21-24bb-4b3c-9876-dc6c6648657e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:17.862916 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602c3e21-24bb-4b3c-9876-dc6c6648657e None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:17.863337 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602c3e21-24bb-4b3c-9876-dc6c6648657e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:17.863797 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602c3e21-24bb-4b3c-9876-dc6c6648657e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:17.875619 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602c3e21-24bb-4b3c-9876-dc6c6648657e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:20:17.881689 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-602c3e21-24bb-4b3c-9876-dc6c6648657e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 06 01:20:17.881689 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:17.881689 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:17.881689 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:17.881689 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:17.881689 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:17.881689 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:17.881689 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:17.881689 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:17.881689 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:17.881689 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:17.881689 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:17.881689 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:20:17.881689 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:20:17.881689 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:20:17.881689 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:20:17.882531 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:20:17.882531 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:20:17.882531 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:20:17.882531 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:20:17.882531 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:17.882531 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:17.882531 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:17.882531 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:17.882531 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:17.882531 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:17.882531 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:17.882531 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:17.882531 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:17.882531 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:17.882531 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:17.882531 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:17.883322 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:17.883322 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:17.883322 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:17.883322 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:17.883322 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:20:17.883322 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:20:17.883322 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:20:17.883322 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:20:17.883322 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:20:17.883322 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:20:17.883322 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 06 01:20:17.883322 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:20:17.886411 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 172/343] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Aug 6 01:20:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:20:17.888891 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bbef6b-be85-4e44-b8e3-6e4835dfb8c7 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:17.889409 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bbef6b-be85-4e44-b8e3-6e4835dfb8c7 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:17.889879 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bbef6b-be85-4e44-b8e3-6e4835dfb8c7 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:17.891767 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 173/344] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:20:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:20:17.892788 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bbef6b-be85-4e44-b8e3-6e4835dfb8c7 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:17.984731 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85a4c41-3dfa-4b4d-916e-632b63fdafce None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:18.021776 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bbef6b-be85-4e44-b8e3-6e4835dfb8c7 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:18.024606 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:18.030300 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fed978-86c6-4577-9258-f46576678a48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:18.031380 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fed978-86c6-4577-9258-f46576678a48 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:18.031913 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fed978-86c6-4577-9258-f46576678a48 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:18.032536 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fed978-86c6-4577-9258-f46576678a48 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:18.052675 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:18.072671 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da61b1e-ddef-4ed5-bdce-c810516a078a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:18.074316 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da61b1e-ddef-4ed5-bdce-c810516a078a None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:18.074997 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da61b1e-ddef-4ed5-bdce-c810516a078a None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:18.075905 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da61b1e-ddef-4ed5-bdce-c810516a078a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:18.092781 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da61b1e-ddef-4ed5-bdce-c810516a078a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:20:18.099721 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-1da61b1e-ddef-4ed5-bdce-c810516a078a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 06 01:20:18.099721 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:18.099721 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:18.099721 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:18.099721 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:18.099721 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:18.099721 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:18.099721 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:18.099721 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:18.099721 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:18.099721 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:18.099721 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:18.099721 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:20:18.099721 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:20:18.099721 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:20:18.099721 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:20:18.100636 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:20:18.100636 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:20:18.100636 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:20:18.100636 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:20:18.100636 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:18.100636 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:18.100636 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:18.100636 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:18.100636 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:18.100636 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:18.100636 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:18.100636 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:18.100636 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:18.100636 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:18.100636 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:18.100636 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:18.101731 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:18.101731 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:18.101731 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:18.101731 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:18.101731 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:20:18.101731 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:20:18.101731 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:20:18.101731 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:20:18.101731 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:20:18.101731 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:20:18.101731 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 06 01:20:18.101731 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:20:18.105232 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 174/345] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Aug 6 01:20:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:20:18.108625 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12602022-a1d0-450a-84af-db9420a87d2b None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:18.109188 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12602022-a1d0-450a-84af-db9420a87d2b None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:18.109805 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12602022-a1d0-450a-84af-db9420a87d2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:18.123432 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fed978-86c6-4577-9258-f46576678a48 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:18.123821 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fed978-86c6-4577-9258-f46576678a48 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:18.140160 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-12602022-a1d0-450a-84af-db9420a87d2b None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:20:18.142378 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-12602022-a1d0-450a-84af-db9420a87d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:20:18.245621 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 175/346] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:20:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:20:18.246287 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12602022-a1d0-450a-84af-db9420a87d2b None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:18.257487 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 172/347] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Aug 6 01:20:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 01:20:18.258323 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fed978-86c6-4577-9258-f46576678a48 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:18.309293 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:18.407730 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12602022-a1d0-450a-84af-db9420a87d2b None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:18.428922 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fed978-86c6-4577-9258-f46576678a48 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:18.433412 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:18.459842 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d4e925-e564-422d-a476-92c437452699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:18.460568 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d4e925-e564-422d-a476-92c437452699 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:18.460924 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d4e925-e564-422d-a476-92c437452699 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:18.461319 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d4e925-e564-422d-a476-92c437452699 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:18.462122 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d4e925-e564-422d-a476-92c437452699 None admin] RBAC: Adding query filter params (name=member) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:18.462559 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d4e925-e564-422d-a476-92c437452699 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:18.464730 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d4e925-e564-422d-a476-92c437452699 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:18.481158 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 176/348] 10.222.0.26 () {56 vars in 1101 bytes} [Sat Aug 6 01:20:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:18.481158 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d4e925-e564-422d-a476-92c437452699 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:18.488273 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176728b6-8b87-4624-a488-720919550e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:18.489173 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176728b6-8b87-4624-a488-720919550e2f None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:18.489603 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176728b6-8b87-4624-a488-720919550e2f None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:18.490024 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176728b6-8b87-4624-a488-720919550e2f None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:18.491549 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176728b6-8b87-4624-a488-720919550e2f None admin] RBAC: Adding query filter params (name=reader) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:18.492007 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176728b6-8b87-4624-a488-720919550e2f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:18.494061 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176728b6-8b87-4624-a488-720919550e2f None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:18.508333 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 173/349] 10.222.0.26 () {56 vars in 1101 bytes} [Sat Aug 6 01:20:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:18.509934 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176728b6-8b87-4624-a488-720919550e2f None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:18.612510 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d4e925-e564-422d-a476-92c437452699 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:18.644761 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:18.655190 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176728b6-8b87-4624-a488-720919550e2f None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:18.664240 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72907f58-fc1a-47ab-a4ff-42b9d8e2f983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:18.665271 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72907f58-fc1a-47ab-a4ff-42b9d8e2f983 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:18.665757 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72907f58-fc1a-47ab-a4ff-42b9d8e2f983 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:18.666244 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72907f58-fc1a-47ab-a4ff-42b9d8e2f983 None admin] PATH_INFO: `/v3/projects/730685c4f62b43c493d42b5897c99a02` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:18.671783 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72907f58-fc1a-47ab-a4ff-42b9d8e2f983 None admin] RBAC: Authorizing `identity:get_project(project_id=730685c4f62b43c493d42b5897c99a02)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:18.673273 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72907f58-fc1a-47ab-a4ff-42b9d8e2f983 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:18.675021 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 177/350] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Aug 6 01:20:18 2022] GET /identity/v3/projects/730685c4f62b43c493d42b5897c99a02 => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:18.675786 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72907f58-fc1a-47ab-a4ff-42b9d8e2f983 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:18.698207 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5886d765-3ce0-408e-9616-b50ddcabf2c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:18.698855 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5886d765-3ce0-408e-9616-b50ddcabf2c4 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:18.699168 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5886d765-3ce0-408e-9616-b50ddcabf2c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:18.699412 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5886d765-3ce0-408e-9616-b50ddcabf2c4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:18.712993 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5886d765-3ce0-408e-9616-b50ddcabf2c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:20:18.718649 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-5886d765-3ce0-408e-9616-b50ddcabf2c4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 06 01:20:18.718649 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:18.718649 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:18.718649 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:18.718649 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:18.718649 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:18.718649 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:18.718649 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:18.718649 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:18.718649 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:18.718649 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:18.718649 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:18.718649 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:20:18.718649 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:20:18.718649 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:20:18.718649 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:20:18.719540 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:20:18.719540 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:20:18.719540 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:20:18.719540 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:20:18.719540 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:18.719540 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:18.719540 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:18.719540 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:18.719540 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:18.719540 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:18.719540 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:18.719540 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:18.719540 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:18.719540 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:18.719540 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:18.719540 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:18.720322 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:18.720322 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:18.720322 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:18.720322 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:18.720322 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:20:18.720322 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:20:18.720322 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:20:18.720322 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:20:18.720322 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:20:18.720322 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:20:18.720322 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 06 01:20:18.720322 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:20:18.720322 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 174/351] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Aug 6 01:20:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:20:18.720998 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5886d765-3ce0-408e-9616-b50ddcabf2c4 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:18.727326 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:18.805952 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72907f58-fc1a-47ab-a4ff-42b9d8e2f983 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:18.848215 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5886d765-3ce0-408e-9616-b50ddcabf2c4 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:18.857550 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0a0839-bbd0-403d-a313-82a5d26df443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:18.858187 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0a0839-bbd0-403d-a313-82a5d26df443 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:18.858466 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0a0839-bbd0-403d-a313-82a5d26df443 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:18.859051 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0a0839-bbd0-403d-a313-82a5d26df443 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:18.900284 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d986c9f-8ed2-4208-b8d9-70fabcedb938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:18.900950 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d986c9f-8ed2-4208-b8d9-70fabcedb938 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:18.901278 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d986c9f-8ed2-4208-b8d9-70fabcedb938 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:18.901557 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d986c9f-8ed2-4208-b8d9-70fabcedb938 None admin] PATH_INFO: `/v3/users/acd8321448b64ce9b1d05e6455cec4c4` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:18.902789 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d986c9f-8ed2-4208-b8d9-70fabcedb938 None admin] RBAC: Authorizing `identity:get_user(user_id=acd8321448b64ce9b1d05e6455cec4c4)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:18.904453 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d986c9f-8ed2-4208-b8d9-70fabcedb938 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:18.906182 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 175/352] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Aug 6 01:20:18 2022] GET /identity/v3/users/acd8321448b64ce9b1d05e6455cec4c4 => generated 293 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:18.909172 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398d7b7f-4062-4d6d-a634-a838a8ec1746 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:18.909172 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398d7b7f-4062-4d6d-a634-a838a8ec1746 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:18.909172 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398d7b7f-4062-4d6d-a634-a838a8ec1746 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:18.913112 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 176/353] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:20:18 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:20:18.913996 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398d7b7f-4062-4d6d-a634-a838a8ec1746 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:18.943277 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:18.948171 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0a0839-bbd0-403d-a313-82a5d26df443 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:18.949917 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0a0839-bbd0-403d-a313-82a5d26df443 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:18.998692 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:19.053941 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398d7b7f-4062-4d6d-a634-a838a8ec1746 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:19.080583 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 178/354] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Aug 6 01:20:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 01:20:19.081546 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0a0839-bbd0-403d-a313-82a5d26df443 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:19.097558 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f677b3-baf9-4a77-a093-715e1d123ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:19.098208 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f677b3-baf9-4a77-a093-715e1d123ba3 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:19.098484 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f677b3-baf9-4a77-a093-715e1d123ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:19.098756 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f677b3-baf9-4a77-a093-715e1d123ba3 None admin] PATH_INFO: `/v3/groups/355b15c08fc7413fb84ea439cfe0c070` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:19.105412 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f677b3-baf9-4a77-a093-715e1d123ba3 None admin] RBAC: Authorizing `identity:get_group(group_id=355b15c08fc7413fb84ea439cfe0c070)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:19.107512 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f677b3-baf9-4a77-a093-715e1d123ba3 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:19.109439 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 177/355] 10.222.0.26 () {56 vars in 1180 bytes} [Sat Aug 6 01:20:18 2022] GET /identity/v3/groups/355b15c08fc7413fb84ea439cfe0c070 => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:19.110541 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f677b3-baf9-4a77-a093-715e1d123ba3 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:19.205154 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0a0839-bbd0-403d-a313-82a5d26df443 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:19.219357 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:19.242836 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f677b3-baf9-4a77-a093-715e1d123ba3 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:19.249750 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0331ef2d-e149-4a62-85ba-0bb10e73107f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:19.250841 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0331ef2d-e149-4a62-85ba-0bb10e73107f None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:19.251125 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0331ef2d-e149-4a62-85ba-0bb10e73107f None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:19.251399 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0331ef2d-e149-4a62-85ba-0bb10e73107f None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:19.252463 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0331ef2d-e149-4a62-85ba-0bb10e73107f None admin] RBAC: Adding query filter params (name=member) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:19.252835 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0331ef2d-e149-4a62-85ba-0bb10e73107f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:19.254860 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0331ef2d-e149-4a62-85ba-0bb10e73107f None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:19.263717 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 179/356] 10.222.0.26 () {56 vars in 1101 bytes} [Sat Aug 6 01:20:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:19.265017 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f098a574-2d81-4c30-a221-db7e75e950bb None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:19.265333 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f098a574-2d81-4c30-a221-db7e75e950bb None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:19.265607 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f098a574-2d81-4c30-a221-db7e75e950bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:19.273847 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:19.285896 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41da49a2-f530-4e2f-9f04-a59843c85983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:19.286522 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41da49a2-f530-4e2f-9f04-a59843c85983 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:19.286806 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41da49a2-f530-4e2f-9f04-a59843c85983 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:19.287079 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41da49a2-f530-4e2f-9f04-a59843c85983 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:19.287873 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41da49a2-f530-4e2f-9f04-a59843c85983 None admin] RBAC: Adding query filter params (role.id=25efb2aed94e4d63b0b6cb8faf5a2ee7, user.id=acd8321448b64ce9b1d05e6455cec4c4, scope.system=all) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:19.288171 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-f098a574-2d81-4c30-a221-db7e75e950bb None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:20:19.288298 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41da49a2-f530-4e2f-9f04-a59843c85983 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:19.289444 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-f098a574-2d81-4c30-a221-db7e75e950bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:20:19.289694 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41da49a2-f530-4e2f-9f04-a59843c85983 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:19.305032 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 178/357] 10.222.0.26 () {56 vars in 1308 bytes} [Sat Aug 6 01:20:19 2022] GET /identity/v3/role_assignments?user.id=acd8321448b64ce9b1d05e6455cec4c4&scope.system=all&role.id=25efb2aed94e4d63b0b6cb8faf5a2ee7 => generated 510 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:19.306174 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41da49a2-f530-4e2f-9f04-a59843c85983 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:19.382065 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 180/358] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:20:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:20:19.389658 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f098a574-2d81-4c30-a221-db7e75e950bb None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:19.437346 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41da49a2-f530-4e2f-9f04-a59843c85983 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:19.490373 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2cfb08-2e70-46a1-b6b5-52bdc44c1cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:19.491118 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2cfb08-2e70-46a1-b6b5-52bdc44c1cd8 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:19.491400 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2cfb08-2e70-46a1-b6b5-52bdc44c1cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:19.492144 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2cfb08-2e70-46a1-b6b5-52bdc44c1cd8 None admin] PATH_INFO: `/v3/projects/730685c4f62b43c493d42b5897c99a02` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:19.493564 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2cfb08-2e70-46a1-b6b5-52bdc44c1cd8 None admin] RBAC: Authorizing `identity:get_project(project_id=730685c4f62b43c493d42b5897c99a02)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:19.495310 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2cfb08-2e70-46a1-b6b5-52bdc44c1cd8 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:19.497471 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 179/359] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Aug 6 01:20:19 2022] GET /identity/v3/projects/730685c4f62b43c493d42b5897c99a02 => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:19.499546 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac29e34-033c-4c7c-9cdc-8f846aa14f78 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:19.499836 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac29e34-033c-4c7c-9cdc-8f846aa14f78 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:19.500110 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac29e34-033c-4c7c-9cdc-8f846aa14f78 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:19.501563 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 180/360] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:20:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:20:19.502104 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac29e34-033c-4c7c-9cdc-8f846aa14f78 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:19.519840 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:19.543007 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f098a574-2d81-4c30-a221-db7e75e950bb None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:19.578178 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:19.585407 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e310a8b-5c05-439e-9873-f75c75d1829e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:19.586096 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e310a8b-5c05-439e-9873-f75c75d1829e None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:19.586321 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e310a8b-5c05-439e-9873-f75c75d1829e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:19.586598 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e310a8b-5c05-439e-9873-f75c75d1829e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:19.587436 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e310a8b-5c05-439e-9873-f75c75d1829e None admin] RBAC: Adding query filter params (group.id=355b15c08fc7413fb84ea439cfe0c070, role.id=6d8b63d350944d78812edbefb32ae8ee, scope.project.id=730685c4f62b43c493d42b5897c99a02) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:19.587758 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e310a8b-5c05-439e-9873-f75c75d1829e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:19.589330 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e310a8b-5c05-439e-9873-f75c75d1829e None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:19.598696 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 181/361] 10.222.0.26 () {56 vars in 1376 bytes} [Sat Aug 6 01:20:19 2022] GET /identity/v3/role_assignments?group.id=355b15c08fc7413fb84ea439cfe0c070&scope.project.id=730685c4f62b43c493d42b5897c99a02&role.id=6d8b63d350944d78812edbefb32ae8ee => generated 260 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:19.599408 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e310a8b-5c05-439e-9873-f75c75d1829e None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:19.633160 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac29e34-033c-4c7c-9cdc-8f846aa14f78 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:19.677125 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4a12a8-5910-44a2-8be2-e85a27b56ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:19.678843 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4a12a8-5910-44a2-8be2-e85a27b56ffc None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:19.678843 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4a12a8-5910-44a2-8be2-e85a27b56ffc None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:19.678843 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4a12a8-5910-44a2-8be2-e85a27b56ffc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:19.751700 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e310a8b-5c05-439e-9873-f75c75d1829e None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:19.776190 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4a12a8-5910-44a2-8be2-e85a27b56ffc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:19.777811 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4a12a8-5910-44a2-8be2-e85a27b56ffc None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:19.796228 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:19.798898 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99abdff2-1b2c-452a-8b52-1e0d5edbe70e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:19.801234 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99abdff2-1b2c-452a-8b52-1e0d5edbe70e None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:19.801234 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99abdff2-1b2c-452a-8b52-1e0d5edbe70e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:19.801234 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99abdff2-1b2c-452a-8b52-1e0d5edbe70e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:19.815753 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99abdff2-1b2c-452a-8b52-1e0d5edbe70e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:20:19.821320 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-99abdff2-1b2c-452a-8b52-1e0d5edbe70e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 01:20:19.821320 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:19.821320 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:19.821320 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:19.821320 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:19.821320 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:19.821320 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:19.821320 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:19.821320 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:19.821320 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:19.821320 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:19.821320 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:19.821320 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:20:19.821320 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:20:19.821320 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:20:19.821320 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:20:19.822211 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:20:19.822211 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:20:19.822211 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:20:19.822211 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:20:19.822211 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:19.822211 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:19.822211 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:19.822211 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:19.822211 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:19.822211 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:19.822211 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:19.822211 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:19.822211 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:19.822211 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:19.822211 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:19.822211 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:19.823007 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:19.823007 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:19.823007 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:19.823007 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:19.823007 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:20:19.823007 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:20:19.823007 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:20:19.823007 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:20:19.823007 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:20:19.823007 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:20:19.823007 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 01:20:19.823007 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:20:19.823007 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 182/362] 10.222.0.26 () {56 vars in 1114 bytes} [Sat Aug 6 01:20:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:20:19.823670 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99abdff2-1b2c-452a-8b52-1e0d5edbe70e None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:19.891621 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:19.921560 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 181/363] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Aug 6 01:20:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 01:20:19.925483 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29964f41-a808-4616-98b1-12af5baa3bd7 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:19.925938 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29964f41-a808-4616-98b1-12af5baa3bd7 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:19.926981 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29964f41-a808-4616-98b1-12af5baa3bd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:19.939893 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99abdff2-1b2c-452a-8b52-1e0d5edbe70e None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:19.947120 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-29964f41-a808-4616-98b1-12af5baa3bd7 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:20:19.948520 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-29964f41-a808-4616-98b1-12af5baa3bd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:20:19.975612 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd514b73-ff06-4ff1-8a3a-f27fa056f051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:19.976536 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd514b73-ff06-4ff1-8a3a-f27fa056f051 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:19.976821 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd514b73-ff06-4ff1-8a3a-f27fa056f051 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:19.977275 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd514b73-ff06-4ff1-8a3a-f27fa056f051 None admin] PATH_INFO: `/v3/groups/355b15c08fc7413fb84ea439cfe0c070` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:19.978456 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd514b73-ff06-4ff1-8a3a-f27fa056f051 None admin] RBAC: Authorizing `identity:get_group(group_id=355b15c08fc7413fb84ea439cfe0c070)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:19.980341 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd514b73-ff06-4ff1-8a3a-f27fa056f051 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:19.983574 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 183/364] 10.222.0.26 () {56 vars in 1180 bytes} [Sat Aug 6 01:20:19 2022] GET /identity/v3/groups/355b15c08fc7413fb84ea439cfe0c070 => generated 226 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:19.984365 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd514b73-ff06-4ff1-8a3a-f27fa056f051 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:20.031182 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 182/365] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:20:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:20:20.033302 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29964f41-a808-4616-98b1-12af5baa3bd7 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:20.078331 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:20.123103 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd514b73-ff06-4ff1-8a3a-f27fa056f051 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:20.149885 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:20.160691 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29964f41-a808-4616-98b1-12af5baa3bd7 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:20.165255 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e5a9d2-f70c-420e-a547-e8ed7f89788c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:20.165889 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e5a9d2-f70c-420e-a547-e8ed7f89788c None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:20.166176 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e5a9d2-f70c-420e-a547-e8ed7f89788c None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:20.166456 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e5a9d2-f70c-420e-a547-e8ed7f89788c None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:20.167182 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e5a9d2-f70c-420e-a547-e8ed7f89788c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:20.167492 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e5a9d2-f70c-420e-a547-e8ed7f89788c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:20.169797 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e5a9d2-f70c-420e-a547-e8ed7f89788c None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:20.181672 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 184/366] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Aug 6 01:20:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:20.183159 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e5a9d2-f70c-420e-a547-e8ed7f89788c None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:20.204000 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74af3c21-3c21-4cc2-b564-1370ca96f08a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:20.204657 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74af3c21-3c21-4cc2-b564-1370ca96f08a None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:20.204941 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74af3c21-3c21-4cc2-b564-1370ca96f08a None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:20.205273 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74af3c21-3c21-4cc2-b564-1370ca96f08a None admin] PATH_INFO: `/v3/projects/0bcfa6bfcd464fc9b0dc2079b24902bc` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:20.208610 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74af3c21-3c21-4cc2-b564-1370ca96f08a None admin] RBAC: Authorizing `identity:get_project(project_id=0bcfa6bfcd464fc9b0dc2079b24902bc)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:20.210501 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74af3c21-3c21-4cc2-b564-1370ca96f08a None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:20.212332 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 183/367] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Aug 6 01:20:20 2022] GET /identity/v3/projects/0bcfa6bfcd464fc9b0dc2079b24902bc => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:20.212543 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74af3c21-3c21-4cc2-b564-1370ca96f08a None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:20.314843 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e5a9d2-f70c-420e-a547-e8ed7f89788c None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:20.345388 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74af3c21-3c21-4cc2-b564-1370ca96f08a None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:20.358204 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e618c74f-d909-4cf8-ad59-f179f6186e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:20.359064 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e618c74f-d909-4cf8-ad59-f179f6186e46 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:20.359497 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e618c74f-d909-4cf8-ad59-f179f6186e46 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:20.359781 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e618c74f-d909-4cf8-ad59-f179f6186e46 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:20.360579 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e618c74f-d909-4cf8-ad59-f179f6186e46 None admin] RBAC: Adding query filter params (group.id=355b15c08fc7413fb84ea439cfe0c070, role.id=61b3b93d37a047e4bb8369b46fe20db9, scope.project.id=730685c4f62b43c493d42b5897c99a02) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:20.360701 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:20.360895 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e618c74f-d909-4cf8-ad59-f179f6186e46 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:20.363806 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e618c74f-d909-4cf8-ad59-f179f6186e46 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:20.370343 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 185/368] 10.222.0.26 () {56 vars in 1376 bytes} [Sat Aug 6 01:20:20 2022] GET /identity/v3/role_assignments?group.id=355b15c08fc7413fb84ea439cfe0c070&scope.project.id=730685c4f62b43c493d42b5897c99a02&role.id=61b3b93d37a047e4bb8369b46fe20db9 => generated 260 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:20.371170 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e618c74f-d909-4cf8-ad59-f179f6186e46 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:20.398240 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bbec79-d05c-4dba-b436-261e7969a6f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:20.399104 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bbec79-d05c-4dba-b436-261e7969a6f2 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:20.399669 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bbec79-d05c-4dba-b436-261e7969a6f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:20.400369 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bbec79-d05c-4dba-b436-261e7969a6f2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:20.417191 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bbec79-d05c-4dba-b436-261e7969a6f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:20:20.423575 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-45bbec79-d05c-4dba-b436-261e7969a6f2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:20:20.423575 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:20.423575 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:20.423575 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:20.423575 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:20.423575 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:20.423575 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:20.423575 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:20.423575 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:20.423575 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:20.423575 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:20.423575 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:20.423575 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:20:20.423575 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:20:20.423575 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:20:20.423575 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:20:20.424526 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:20:20.424526 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:20:20.424526 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:20:20.424526 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:20:20.424526 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:20.424526 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:20.424526 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:20.424526 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:20.424526 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:20.424526 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:20.424526 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:20.424526 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:20.424526 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:20.424526 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:20.424526 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:20.424526 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:20.425327 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:20.425327 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:20.425327 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:20.425327 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:20.425327 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:20:20.425327 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:20:20.425327 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:20:20.425327 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:20:20.425327 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:20:20.425327 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:20:20.425327 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:20:20.425327 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:20:20.427973 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 184/369] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Aug 6 01:20:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:20:20.429200 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bbec79-d05c-4dba-b436-261e7969a6f2 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:20.442029 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:20.514305 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e618c74f-d909-4cf8-ad59-f179f6186e46 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:20.555293 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c16b46-af21-4ffb-b7f8-4429eb2c6e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:20.556272 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c16b46-af21-4ffb-b7f8-4429eb2c6e1e None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:20.556777 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c16b46-af21-4ffb-b7f8-4429eb2c6e1e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:20.557296 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c16b46-af21-4ffb-b7f8-4429eb2c6e1e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:20.575346 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bbec79-d05c-4dba-b436-261e7969a6f2 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:20.617676 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17f6895-b5e5-4055-8622-d71aabcf1105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:20.618326 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17f6895-b5e5-4055-8622-d71aabcf1105 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:20.618609 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17f6895-b5e5-4055-8622-d71aabcf1105 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:20.618906 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17f6895-b5e5-4055-8622-d71aabcf1105 None admin] PATH_INFO: `/v3/groups/355b15c08fc7413fb84ea439cfe0c070` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:20.620676 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17f6895-b5e5-4055-8622-d71aabcf1105 None admin] RBAC: Authorizing `identity:get_group(group_id=355b15c08fc7413fb84ea439cfe0c070)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:20.623307 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17f6895-b5e5-4055-8622-d71aabcf1105 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:20.626174 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 185/370] 10.222.0.26 () {56 vars in 1180 bytes} [Sat Aug 6 01:20:20 2022] GET /identity/v3/groups/355b15c08fc7413fb84ea439cfe0c070 => generated 226 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:20.627671 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17f6895-b5e5-4055-8622-d71aabcf1105 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:20.659790 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c16b46-af21-4ffb-b7f8-4429eb2c6e1e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:20.661456 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c16b46-af21-4ffb-b7f8-4429eb2c6e1e None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:20.683759 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:20.723334 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:20.763888 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17f6895-b5e5-4055-8622-d71aabcf1105 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:20.799516 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 186/371] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Aug 6 01:20:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 01:20:20.801073 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c16b46-af21-4ffb-b7f8-4429eb2c6e1e None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:20.805641 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a813d3-1cc2-4354-97a3-c80ac6b5a08d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:20.806275 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a813d3-1cc2-4354-97a3-c80ac6b5a08d None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:20.806557 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a813d3-1cc2-4354-97a3-c80ac6b5a08d None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:20.806846 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a813d3-1cc2-4354-97a3-c80ac6b5a08d None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:20.807550 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a813d3-1cc2-4354-97a3-c80ac6b5a08d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:20.808026 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a813d3-1cc2-4354-97a3-c80ac6b5a08d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:20.810132 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a813d3-1cc2-4354-97a3-c80ac6b5a08d None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:20.818931 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 186/372] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Aug 6 01:20:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:20.819751 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a813d3-1cc2-4354-97a3-c80ac6b5a08d None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:20.943893 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c16b46-af21-4ffb-b7f8-4429eb2c6e1e None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:20.949151 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a813d3-1cc2-4354-97a3-c80ac6b5a08d None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:20.953428 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:20.990726 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f845dd94-7007-4730-968b-2a4a529a51b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:20.991000 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfefed5e-4236-4fa8-869e-a92081987277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:20.991802 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfefed5e-4236-4fa8-869e-a92081987277 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:20.992016 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f845dd94-7007-4730-968b-2a4a529a51b9 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:20.992162 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfefed5e-4236-4fa8-869e-a92081987277 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:20.992543 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfefed5e-4236-4fa8-869e-a92081987277 None admin] PATH_INFO: `/v3/projects/0bcfa6bfcd464fc9b0dc2079b24902bc` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:20.992809 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f845dd94-7007-4730-968b-2a4a529a51b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:20.993287 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f845dd94-7007-4730-968b-2a4a529a51b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:20.994239 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f845dd94-7007-4730-968b-2a4a529a51b9 None admin] RBAC: Adding query filter params (group.id=355b15c08fc7413fb84ea439cfe0c070, role.id=61b3b93d37a047e4bb8369b46fe20db9, scope.project.id=0bcfa6bfcd464fc9b0dc2079b24902bc) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:20.994727 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f845dd94-7007-4730-968b-2a4a529a51b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:20.994890 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfefed5e-4236-4fa8-869e-a92081987277 None admin] RBAC: Authorizing `identity:get_project(project_id=0bcfa6bfcd464fc9b0dc2079b24902bc)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:20.996533 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f845dd94-7007-4730-968b-2a4a529a51b9 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:20.996643 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfefed5e-4236-4fa8-869e-a92081987277 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:20.998932 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 187/373] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Aug 6 01:20:20 2022] GET /identity/v3/projects/0bcfa6bfcd464fc9b0dc2079b24902bc => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:20.999646 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfefed5e-4236-4fa8-869e-a92081987277 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:21.003781 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 187/374] 10.222.0.26 () {56 vars in 1376 bytes} [Sat Aug 6 01:20:20 2022] GET /identity/v3/role_assignments?group.id=355b15c08fc7413fb84ea439cfe0c070&scope.project.id=0bcfa6bfcd464fc9b0dc2079b24902bc&role.id=61b3b93d37a047e4bb8369b46fe20db9 => generated 260 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:21.005276 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f845dd94-7007-4730-968b-2a4a529a51b9 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:21.039899 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:21.201616 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfefed5e-4236-4fa8-869e-a92081987277 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:21.205804 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f845dd94-7007-4730-968b-2a4a529a51b9 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:21.247733 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1875edc-7063-4aaf-b4e0-85f61610e01e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:21.248373 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1875edc-7063-4aaf-b4e0-85f61610e01e None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:21.248704 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1875edc-7063-4aaf-b4e0-85f61610e01e None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:21.248993 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1875edc-7063-4aaf-b4e0-85f61610e01e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:21.252930 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c90fba-ac92-426b-8ab1-2c1235873565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:21.253992 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c90fba-ac92-426b-8ab1-2c1235873565 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:21.254442 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c90fba-ac92-426b-8ab1-2c1235873565 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:21.254863 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c90fba-ac92-426b-8ab1-2c1235873565 None admin] PATH_INFO: `/v3/groups/355b15c08fc7413fb84ea439cfe0c070` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:21.256197 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c90fba-ac92-426b-8ab1-2c1235873565 None admin] RBAC: Authorizing `identity:get_group(group_id=355b15c08fc7413fb84ea439cfe0c070)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:21.258830 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c90fba-ac92-426b-8ab1-2c1235873565 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:21.261166 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 188/375] 10.222.0.26 () {56 vars in 1180 bytes} [Sat Aug 6 01:20:21 2022] GET /identity/v3/groups/355b15c08fc7413fb84ea439cfe0c070 => generated 226 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:21.270051 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c90fba-ac92-426b-8ab1-2c1235873565 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:21.314530 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:21.372930 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1875edc-7063-4aaf-b4e0-85f61610e01e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:21.374421 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1875edc-7063-4aaf-b4e0-85f61610e01e None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:21.442237 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:21.484302 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c90fba-ac92-426b-8ab1-2c1235873565 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:21.503206 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 188/376] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Aug 6 01:20:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 504 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 01:20:21.508738 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1875edc-7063-4aaf-b4e0-85f61610e01e None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:21.562326 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5230ab-9340-4816-a542-954717a6e958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:21.563327 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5230ab-9340-4816-a542-954717a6e958 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:21.563871 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5230ab-9340-4816-a542-954717a6e958 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:21.564354 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5230ab-9340-4816-a542-954717a6e958 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:21.565400 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5230ab-9340-4816-a542-954717a6e958 None admin] RBAC: Adding query filter params (group.id=355b15c08fc7413fb84ea439cfe0c070, role.id=6d8b63d350944d78812edbefb32ae8ee, scope.project.id=0bcfa6bfcd464fc9b0dc2079b24902bc) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:21.565926 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5230ab-9340-4816-a542-954717a6e958 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:21.567574 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5230ab-9340-4816-a542-954717a6e958 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:21.580124 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 189/377] 10.222.0.26 () {56 vars in 1376 bytes} [Sat Aug 6 01:20:21 2022] GET /identity/v3/role_assignments?group.id=355b15c08fc7413fb84ea439cfe0c070&scope.project.id=0bcfa6bfcd464fc9b0dc2079b24902bc&role.id=6d8b63d350944d78812edbefb32ae8ee => generated 260 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:21.584713 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:21.623167 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1875edc-7063-4aaf-b4e0-85f61610e01e None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:21.661763 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73f3871-576d-4dc6-8940-0b76dae9c9b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:21.662410 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73f3871-576d-4dc6-8940-0b76dae9c9b2 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:21.662693 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73f3871-576d-4dc6-8940-0b76dae9c9b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:21.662966 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73f3871-576d-4dc6-8940-0b76dae9c9b2 None admin] PATH_INFO: `/v3/projects/477e329599354f58af86e800a455a8b2` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:21.664149 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73f3871-576d-4dc6-8940-0b76dae9c9b2 None admin] RBAC: Authorizing `identity:get_project(project_id=477e329599354f58af86e800a455a8b2)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:21.665621 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73f3871-576d-4dc6-8940-0b76dae9c9b2 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:21.667673 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 189/378] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Aug 6 01:20:21 2022] GET /identity/v3/projects/477e329599354f58af86e800a455a8b2 => generated 344 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:20:21.673528 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5230ab-9340-4816-a542-954717a6e958 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:21.821620 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5230ab-9340-4816-a542-954717a6e958 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:21.841507 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69d7f00-9ff6-41d8-abe7-105d7332fc0a None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:21.841829 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69d7f00-9ff6-41d8-abe7-105d7332fc0a None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:21.842062 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69d7f00-9ff6-41d8-abe7-105d7332fc0a None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:21.843688 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 190/379] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:20:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:20:21.849284 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf283662-9ad3-4978-be59-ceb0dba67f35 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:21.849573 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf283662-9ad3-4978-be59-ceb0dba67f35 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:21.849857 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf283662-9ad3-4978-be59-ceb0dba67f35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:21.862715 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27ff103-31a8-4b00-863a-0fce8ff46ab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:21.863372 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27ff103-31a8-4b00-863a-0fce8ff46ab4 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:21.863656 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27ff103-31a8-4b00-863a-0fce8ff46ab4 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:21.863938 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27ff103-31a8-4b00-863a-0fce8ff46ab4 None admin] PATH_INFO: `/v3/groups/f93e7761d988449a9be67baf7457c4cb` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:21.871381 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27ff103-31a8-4b00-863a-0fce8ff46ab4 None admin] RBAC: Authorizing `identity:get_group(group_id=f93e7761d988449a9be67baf7457c4cb)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:21.872556 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-bf283662-9ad3-4978-be59-ceb0dba67f35 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:20:21.873421 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27ff103-31a8-4b00-863a-0fce8ff46ab4 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:21.873782 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-bf283662-9ad3-4978-be59-ceb0dba67f35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:20:21.880007 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 190/380] 10.222.0.26 () {56 vars in 1180 bytes} [Sat Aug 6 01:20:21 2022] GET /identity/v3/groups/f93e7761d988449a9be67baf7457c4cb => generated 229 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:21.880978 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27ff103-31a8-4b00-863a-0fce8ff46ab4 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:21.910812 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:22.012958 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 191/381] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:20:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 01:20:22.035740 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf283662-9ad3-4978-be59-ceb0dba67f35 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:22.065183 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27ff103-31a8-4b00-863a-0fce8ff46ab4 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:22.104495 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc2066b-369e-45ca-bdea-5946656b64cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:22.106353 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc2066b-369e-45ca-bdea-5946656b64cb None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:22.106639 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc2066b-369e-45ca-bdea-5946656b64cb None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:22.106918 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc2066b-369e-45ca-bdea-5946656b64cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:22.107718 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc2066b-369e-45ca-bdea-5946656b64cb None admin] RBAC: Adding query filter params (group.id=f93e7761d988449a9be67baf7457c4cb, role.id=4559b9a0837249f895a13c70096f8d38, scope.project.id=477e329599354f58af86e800a455a8b2) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:22.108035 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc2066b-369e-45ca-bdea-5946656b64cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:22.109513 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc2066b-369e-45ca-bdea-5946656b64cb None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:22.126525 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 191/382] 10.222.0.26 () {56 vars in 1376 bytes} [Sat Aug 6 01:20:21 2022] GET /identity/v3/role_assignments?group.id=f93e7761d988449a9be67baf7457c4cb&scope.project.id=477e329599354f58af86e800a455a8b2&role.id=4559b9a0837249f895a13c70096f8d38 => generated 260 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:20:22.167704 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:22.222961 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf283662-9ad3-4978-be59-ceb0dba67f35 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:22.278887 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50694daf-3754-49c1-94a8-e2b7e5d16bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:22.279925 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50694daf-3754-49c1-94a8-e2b7e5d16bfc None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:22.280426 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50694daf-3754-49c1-94a8-e2b7e5d16bfc None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:22.281042 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50694daf-3754-49c1-94a8-e2b7e5d16bfc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:22.298277 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50694daf-3754-49c1-94a8-e2b7e5d16bfc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:20:22.303557 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-50694daf-3754-49c1-94a8-e2b7e5d16bfc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 01:20:22.303557 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:22.303557 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:22.303557 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:22.303557 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:22.303557 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:22.303557 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:22.303557 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:22.303557 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:22.303557 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:22.303557 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:22.303557 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:22.303557 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:20:22.303557 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:20:22.303557 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:20:22.303557 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:20:22.304456 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:20:22.304456 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:20:22.304456 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:20:22.304456 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:20:22.304456 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:22.304456 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:22.304456 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:22.304456 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:22.304456 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:22.304456 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:22.304456 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:22.304456 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:22.304456 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:22.304456 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:22.304456 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:22.304456 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:22.305324 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:22.305324 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:22.305324 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:22.305324 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:22.305324 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:20:22.305324 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:20:22.305324 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:20:22.305324 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:20:22.305324 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:20:22.305324 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:20:22.305324 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 01:20:22.305324 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:20:22.308102 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 192/383] 10.222.0.26 () {56 vars in 1114 bytes} [Sat Aug 6 01:20:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:20:22.313711 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc2066b-369e-45ca-bdea-5946656b64cb None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:22.426689 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:22.460124 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc2066b-369e-45ca-bdea-5946656b64cb None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:22.508924 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8199c2d1-75f4-4156-8214-b6f8968fbb35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:22.509914 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8199c2d1-75f4-4156-8214-b6f8968fbb35 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:22.510425 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8199c2d1-75f4-4156-8214-b6f8968fbb35 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:22.511021 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8199c2d1-75f4-4156-8214-b6f8968fbb35 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:22.511983 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8199c2d1-75f4-4156-8214-b6f8968fbb35 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:22.512997 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8199c2d1-75f4-4156-8214-b6f8968fbb35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:22.514617 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8199c2d1-75f4-4156-8214-b6f8968fbb35 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:22.528662 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 192/384] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Aug 6 01:20:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:22.533884 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50694daf-3754-49c1-94a8-e2b7e5d16bfc None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:22.711337 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50694daf-3754-49c1-94a8-e2b7e5d16bfc None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:22.745997 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:22.756457 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afebefb-aab3-433f-9f3e-580b8d59e443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:22.757660 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afebefb-aab3-433f-9f3e-580b8d59e443 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:22.758247 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afebefb-aab3-433f-9f3e-580b8d59e443 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:22.758882 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afebefb-aab3-433f-9f3e-580b8d59e443 None admin] PATH_INFO: `/v3/groups/355b15c08fc7413fb84ea439cfe0c070` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:22.760530 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afebefb-aab3-433f-9f3e-580b8d59e443 None admin] RBAC: Authorizing `identity:get_group(group_id=355b15c08fc7413fb84ea439cfe0c070)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:22.762031 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afebefb-aab3-433f-9f3e-580b8d59e443 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:22.764836 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 193/385] 10.222.0.26 () {56 vars in 1180 bytes} [Sat Aug 6 01:20:22 2022] GET /identity/v3/groups/355b15c08fc7413fb84ea439cfe0c070 => generated 226 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:22.770120 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8199c2d1-75f4-4156-8214-b6f8968fbb35 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:22.939390 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8199c2d1-75f4-4156-8214-b6f8968fbb35 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:22.980791 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd1c50b-6636-4f8f-9454-11f302c30856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:22.983640 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd1c50b-6636-4f8f-9454-11f302c30856 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:22.983928 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd1c50b-6636-4f8f-9454-11f302c30856 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:22.984210 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd1c50b-6636-4f8f-9454-11f302c30856 None admin] PATH_INFO: `/v3/projects/730685c4f62b43c493d42b5897c99a02` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:22.985462 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd1c50b-6636-4f8f-9454-11f302c30856 None admin] RBAC: Authorizing `identity:get_project(project_id=730685c4f62b43c493d42b5897c99a02)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:22.986913 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd1c50b-6636-4f8f-9454-11f302c30856 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:22.988742 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 193/386] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Aug 6 01:20:22 2022] GET /identity/v3/projects/730685c4f62b43c493d42b5897c99a02 => generated 298 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:22.994556 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afebefb-aab3-433f-9f3e-580b8d59e443 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:23.078858 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0842747d-d3a7-4f98-aaf1-72e5508053b9 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:23.079196 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0842747d-d3a7-4f98-aaf1-72e5508053b9 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:23.079432 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0842747d-d3a7-4f98-aaf1-72e5508053b9 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:23.080794 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 194/387] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:20:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:20:23.087683 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bddbba-a45c-4820-b398-6c47eaff41da None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:23.087975 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bddbba-a45c-4820-b398-6c47eaff41da None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:23.088258 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bddbba-a45c-4820-b398-6c47eaff41da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:23.097380 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:23.114899 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-93bddbba-a45c-4820-b398-6c47eaff41da None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:20:23.116107 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-93bddbba-a45c-4820-b398-6c47eaff41da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:20:23.133233 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afebefb-aab3-433f-9f3e-580b8d59e443 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:23.181493 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e5bf05-6423-431e-ad52-ff516b19fa08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:23.182434 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e5bf05-6423-431e-ad52-ff516b19fa08 None admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:23.182783 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e5bf05-6423-431e-ad52-ff516b19fa08 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:23.183072 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e5bf05-6423-431e-ad52-ff516b19fa08 None admin] PATH_INFO: `/v3/projects/730685c4f62b43c493d42b5897c99a02/groups/355b15c08fc7413fb84ea439cfe0c070/roles/6d8b63d350944d78812edbefb32ae8ee` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:23.185568 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e5bf05-6423-431e-ad52-ff516b19fa08 None admin] RBAC: Authorizing `identity:create_grant(project_id=730685c4f62b43c493d42b5897c99a02, group_id=355b15c08fc7413fb84ea439cfe0c070, role_id=6d8b63d350944d78812edbefb32ae8ee)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:23.187539 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e5bf05-6423-431e-ad52-ff516b19fa08 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:23.205270 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 194/388] 10.222.0.26 () {58 vars in 1442 bytes} [Sat Aug 6 01:20:22 2022] PUT /identity/v3/projects/730685c4f62b43c493d42b5897c99a02/groups/355b15c08fc7413fb84ea439cfe0c070/roles/6d8b63d350944d78812edbefb32ae8ee => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:20:23.265658 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 195/389] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:20:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:20:23.300693 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e5bf05-6423-431e-ad52-ff516b19fa08 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:23.393638 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:23.448388 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e5bf05-6423-431e-ad52-ff516b19fa08 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:23.487539 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca76c12-b7f8-4e59-89c2-1ddd5b40b1bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:23.488195 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca76c12-b7f8-4e59-89c2-1ddd5b40b1bf None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:23.488480 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca76c12-b7f8-4e59-89c2-1ddd5b40b1bf None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:23.488760 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca76c12-b7f8-4e59-89c2-1ddd5b40b1bf None admin] PATH_INFO: `/v3/roles/member` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:23.500791 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca76c12-b7f8-4e59-89c2-1ddd5b40b1bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:20:23.505889 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-8ca76c12-b7f8-4e59-89c2-1ddd5b40b1bf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 06 01:20:23.505889 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:23.505889 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:23.505889 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:23.505889 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:23.505889 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:23.505889 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:23.505889 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:23.505889 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:23.505889 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:23.505889 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:23.505889 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:23.505889 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:20:23.505889 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:20:23.505889 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:20:23.505889 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:20:23.506751 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:20:23.506751 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:20:23.506751 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:20:23.506751 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:20:23.506751 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:23.506751 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:23.506751 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:23.506751 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:23.506751 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:23.506751 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:23.506751 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:23.506751 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:23.506751 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:23.506751 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:23.506751 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:23.506751 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:23.507542 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:23.507542 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:23.507542 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:23.507542 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:23.507542 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:20:23.507542 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:20:23.507542 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:20:23.507542 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:20:23.507542 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:20:23.507542 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:20:23.507542 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 06 01:20:23.507542 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:20:23.507542 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 195/390] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Aug 6 01:20:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:20:23.514133 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bddbba-a45c-4820-b398-6c47eaff41da None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:23.564276 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8fc08a-64b5-467f-ab38-55be6a2a10e4 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:23.564769 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8fc08a-64b5-467f-ab38-55be6a2a10e4 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:23.564769 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8fc08a-64b5-467f-ab38-55be6a2a10e4 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:23.566190 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 196/391] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:20:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:20:23.574366 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d0ed3f-2131-4f33-ac0b-0943e4362067 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:23.575212 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d0ed3f-2131-4f33-ac0b-0943e4362067 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:23.575709 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d0ed3f-2131-4f33-ac0b-0943e4362067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:23.622653 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-40d0ed3f-2131-4f33-ac0b-0943e4362067 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:20:23.624202 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-40d0ed3f-2131-4f33-ac0b-0943e4362067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:20:23.637573 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bddbba-a45c-4820-b398-6c47eaff41da None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:23.681073 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1c6332-ea23-4c1d-a918-2d9f78bd55f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:23.681073 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1c6332-ea23-4c1d-a918-2d9f78bd55f9 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:23.681073 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1c6332-ea23-4c1d-a918-2d9f78bd55f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:23.681073 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1c6332-ea23-4c1d-a918-2d9f78bd55f9 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:23.681496 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1c6332-ea23-4c1d-a918-2d9f78bd55f9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:23.681496 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1c6332-ea23-4c1d-a918-2d9f78bd55f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:23.681496 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1c6332-ea23-4c1d-a918-2d9f78bd55f9 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:23.684785 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:23.704655 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 196/392] 10.222.0.26 () {56 vars in 1101 bytes} [Sat Aug 6 01:20:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:23.705963 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1c6332-ea23-4c1d-a918-2d9f78bd55f9 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:23.744669 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 197/393] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:20:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:20:23.782144 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d0ed3f-2131-4f33-ac0b-0943e4362067 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:23.811864 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1c6332-ea23-4c1d-a918-2d9f78bd55f9 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:23.847275 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6223068b-40d8-4a5e-8456-d5716400f7a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:23.847852 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6223068b-40d8-4a5e-8456-d5716400f7a3 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:23.848140 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6223068b-40d8-4a5e-8456-d5716400f7a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:23.848422 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6223068b-40d8-4a5e-8456-d5716400f7a3 None admin] PATH_INFO: `/v3/groups/355b15c08fc7413fb84ea439cfe0c070` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:23.849636 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6223068b-40d8-4a5e-8456-d5716400f7a3 None admin] RBAC: Authorizing `identity:get_group(group_id=355b15c08fc7413fb84ea439cfe0c070)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:23.851330 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6223068b-40d8-4a5e-8456-d5716400f7a3 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:23.857325 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 197/394] 10.222.0.26 () {56 vars in 1180 bytes} [Sat Aug 6 01:20:23 2022] GET /identity/v3/groups/355b15c08fc7413fb84ea439cfe0c070 => generated 226 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:23.860734 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6223068b-40d8-4a5e-8456-d5716400f7a3 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:23.871052 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:23.926273 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:23.957356 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d0ed3f-2131-4f33-ac0b-0943e4362067 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:23.995026 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b13968f-71cf-4070-be98-0fa0162e9a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:23.995727 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b13968f-71cf-4070-be98-0fa0162e9a3d None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:23.996058 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b13968f-71cf-4070-be98-0fa0162e9a3d None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:23.996406 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b13968f-71cf-4070-be98-0fa0162e9a3d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:24.001442 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6223068b-40d8-4a5e-8456-d5716400f7a3 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:24.017435 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b13968f-71cf-4070-be98-0fa0162e9a3d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:20:24.024990 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-4b13968f-71cf-4070-be98-0fa0162e9a3d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 06 01:20:24.024990 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:24.024990 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:24.024990 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:24.024990 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:24.024990 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:24.024990 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:24.024990 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:24.024990 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:24.024990 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:24.024990 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:24.024990 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:24.024990 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:20:24.024990 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:20:24.024990 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:20:24.024990 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:20:24.026071 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:20:24.026071 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:20:24.026071 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:20:24.026071 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:20:24.026071 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:24.026071 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:24.026071 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:24.026071 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:24.026071 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:24.026071 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:24.026071 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:24.026071 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:24.026071 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:24.026071 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:24.026071 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:24.026071 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:24.027044 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:24.027044 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:24.027044 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:24.027044 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:24.027044 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:20:24.027044 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:20:24.027044 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:20:24.027044 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:20:24.027044 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:20:24.027044 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:20:24.027044 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 06 01:20:24.027044 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:20:24.030940 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 198/395] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Aug 6 01:20:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:20:24.032597 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d5c05a-33dd-42a4-bf0f-3ed55b0ea3fe None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:24.033462 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d5c05a-33dd-42a4-bf0f-3ed55b0ea3fe None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:24.033896 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d5c05a-33dd-42a4-bf0f-3ed55b0ea3fe None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:24.039517 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 199/396] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:20:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:20:24.039517 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d5c05a-33dd-42a4-bf0f-3ed55b0ea3fe None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:24.046402 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eb669e-4178-4ea5-911e-2dd0dd2240db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:24.047324 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eb669e-4178-4ea5-911e-2dd0dd2240db None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:24.047799 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eb669e-4178-4ea5-911e-2dd0dd2240db None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:24.048298 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eb669e-4178-4ea5-911e-2dd0dd2240db None admin] PATH_INFO: `/v3/projects/730685c4f62b43c493d42b5897c99a02` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:24.049737 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72eb669e-4178-4ea5-911e-2dd0dd2240db None admin] RBAC: Authorizing `identity:get_project(project_id=730685c4f62b43c493d42b5897c99a02)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:24.051365 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72eb669e-4178-4ea5-911e-2dd0dd2240db None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:24.053522 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 198/397] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Aug 6 01:20:23 2022] GET /identity/v3/projects/730685c4f62b43c493d42b5897c99a02 => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:24.054832 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950c1b91-e765-4cbb-bfd3-cb1162bf5d53 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:24.055318 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950c1b91-e765-4cbb-bfd3-cb1162bf5d53 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:24.055783 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950c1b91-e765-4cbb-bfd3-cb1162bf5d53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:24.074385 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-950c1b91-e765-4cbb-bfd3-cb1162bf5d53 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:20:24.075842 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-950c1b91-e765-4cbb-bfd3-cb1162bf5d53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:20:24.147691 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 199/398] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:20:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:20:24.154189 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950c1b91-e765-4cbb-bfd3-cb1162bf5d53 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:24.159546 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d5c05a-33dd-42a4-bf0f-3ed55b0ea3fe None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:24.169070 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:24.198210 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e938f840-c5e0-47f2-b05d-b35fb1f201b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:24.198846 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e938f840-c5e0-47f2-b05d-b35fb1f201b8 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:24.199130 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e938f840-c5e0-47f2-b05d-b35fb1f201b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:24.199995 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e938f840-c5e0-47f2-b05d-b35fb1f201b8 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:24.201031 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e938f840-c5e0-47f2-b05d-b35fb1f201b8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:24.201192 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e938f840-c5e0-47f2-b05d-b35fb1f201b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:24.202641 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e938f840-c5e0-47f2-b05d-b35fb1f201b8 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:24.212641 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 200/399] 10.222.0.26 () {56 vars in 1101 bytes} [Sat Aug 6 01:20:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:24.213642 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e938f840-c5e0-47f2-b05d-b35fb1f201b8 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:24.237331 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:24.278163 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950c1b91-e765-4cbb-bfd3-cb1162bf5d53 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:24.312872 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc27a515-d6cc-4271-8561-7c6feef0152a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:24.313843 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc27a515-d6cc-4271-8561-7c6feef0152a None admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:24.314291 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc27a515-d6cc-4271-8561-7c6feef0152a None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:24.314706 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc27a515-d6cc-4271-8561-7c6feef0152a None admin] PATH_INFO: `/v3/projects/730685c4f62b43c493d42b5897c99a02/groups/355b15c08fc7413fb84ea439cfe0c070/roles/61b3b93d37a047e4bb8369b46fe20db9` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:24.316801 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc27a515-d6cc-4271-8561-7c6feef0152a None admin] RBAC: Authorizing `identity:create_grant(project_id=730685c4f62b43c493d42b5897c99a02, group_id=355b15c08fc7413fb84ea439cfe0c070, role_id=61b3b93d37a047e4bb8369b46fe20db9)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:24.318392 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc27a515-d6cc-4271-8561-7c6feef0152a None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:24.321261 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e938f840-c5e0-47f2-b05d-b35fb1f201b8 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:24.333026 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 200/400] 10.222.0.26 () {58 vars in 1442 bytes} [Sat Aug 6 01:20:24 2022] PUT /identity/v3/projects/730685c4f62b43c493d42b5897c99a02/groups/355b15c08fc7413fb84ea439cfe0c070/roles/61b3b93d37a047e4bb8369b46fe20db9 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:20:24.334155 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc27a515-d6cc-4271-8561-7c6feef0152a None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:24.388248 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc6b1f0-023d-41bd-b0f9-c4b996ac170a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:24.389262 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc6b1f0-023d-41bd-b0f9-c4b996ac170a None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:24.389685 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc6b1f0-023d-41bd-b0f9-c4b996ac170a None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:24.391090 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc6b1f0-023d-41bd-b0f9-c4b996ac170a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:24.408729 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc6b1f0-023d-41bd-b0f9-c4b996ac170a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:20:24.414539 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-bcc6b1f0-023d-41bd-b0f9-c4b996ac170a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 01:20:24.414539 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:24.414539 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:24.414539 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:24.414539 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:24.414539 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:24.414539 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:24.414539 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:24.414539 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:24.414539 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:24.414539 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:24.414539 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:24.414539 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:20:24.414539 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:20:24.414539 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:20:24.414539 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:20:24.415494 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:20:24.415494 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:20:24.415494 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:20:24.415494 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:20:24.415494 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:24.415494 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:24.415494 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:24.415494 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:24.415494 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:24.415494 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:24.415494 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:24.415494 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:24.415494 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:24.415494 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:24.415494 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:24.415494 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:24.416280 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:24.416280 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:24.416280 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:24.416280 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:24.416280 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:20:24.416280 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:20:24.416280 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:20:24.416280 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:20:24.416280 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:20:24.416280 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:20:24.416280 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 01:20:24.416280 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:20:24.424680 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 201/401] 10.222.0.26 () {56 vars in 1114 bytes} [Sat Aug 6 01:20:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:20:24.425905 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc6b1f0-023d-41bd-b0f9-c4b996ac170a None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:24.459356 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:24.544889 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc27a515-d6cc-4271-8561-7c6feef0152a None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:24.569125 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc6b1f0-023d-41bd-b0f9-c4b996ac170a None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:24.570157 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:24.583896 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a2a756-675f-402e-8e08-3f5f2a9e7c28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:24.584762 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a2a756-675f-402e-8e08-3f5f2a9e7c28 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:24.584880 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a2a756-675f-402e-8e08-3f5f2a9e7c28 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:24.585127 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a2a756-675f-402e-8e08-3f5f2a9e7c28 None admin] PATH_INFO: `/v3/groups/355b15c08fc7413fb84ea439cfe0c070` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:24.586387 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a2a756-675f-402e-8e08-3f5f2a9e7c28 None admin] RBAC: Authorizing `identity:get_group(group_id=355b15c08fc7413fb84ea439cfe0c070)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:24.587816 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a2a756-675f-402e-8e08-3f5f2a9e7c28 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:24.590887 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 201/402] 10.222.0.26 () {56 vars in 1180 bytes} [Sat Aug 6 01:20:24 2022] GET /identity/v3/groups/355b15c08fc7413fb84ea439cfe0c070 => generated 226 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:24.596282 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a2a756-675f-402e-8e08-3f5f2a9e7c28 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:24.615201 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3639e3d8-22fd-4952-8cc0-55e991b738ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:24.616131 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3639e3d8-22fd-4952-8cc0-55e991b738ab None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:24.616560 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3639e3d8-22fd-4952-8cc0-55e991b738ab None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:24.617135 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3639e3d8-22fd-4952-8cc0-55e991b738ab None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:24.618305 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3639e3d8-22fd-4952-8cc0-55e991b738ab None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:24.618762 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3639e3d8-22fd-4952-8cc0-55e991b738ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:24.620394 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3639e3d8-22fd-4952-8cc0-55e991b738ab None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:24.633589 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 202/403] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Aug 6 01:20:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:24.641134 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3639e3d8-22fd-4952-8cc0-55e991b738ab None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:24.739607 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:24.755171 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3639e3d8-22fd-4952-8cc0-55e991b738ab None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:24.764770 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a2a756-675f-402e-8e08-3f5f2a9e7c28 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:24.797537 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42db199-bb86-4082-8b00-51c1a3b6afcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:24.798585 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42db199-bb86-4082-8b00-51c1a3b6afcb None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:24.799079 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42db199-bb86-4082-8b00-51c1a3b6afcb None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:24.799589 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42db199-bb86-4082-8b00-51c1a3b6afcb None admin] PATH_INFO: `/v3/groups/355b15c08fc7413fb84ea439cfe0c070` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:24.800849 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0693c4bf-9ec8-4514-8f3b-42c92e1b9f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:24.801230 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42db199-bb86-4082-8b00-51c1a3b6afcb None admin] RBAC: Authorizing `identity:get_group(group_id=355b15c08fc7413fb84ea439cfe0c070)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:24.801530 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0693c4bf-9ec8-4514-8f3b-42c92e1b9f69 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:24.801816 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0693c4bf-9ec8-4514-8f3b-42c92e1b9f69 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:24.802153 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0693c4bf-9ec8-4514-8f3b-42c92e1b9f69 None admin] PATH_INFO: `/v3/projects/0bcfa6bfcd464fc9b0dc2079b24902bc` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:24.803336 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0693c4bf-9ec8-4514-8f3b-42c92e1b9f69 None admin] RBAC: Authorizing `identity:get_project(project_id=0bcfa6bfcd464fc9b0dc2079b24902bc)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:24.803707 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42db199-bb86-4082-8b00-51c1a3b6afcb None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:24.804815 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0693c4bf-9ec8-4514-8f3b-42c92e1b9f69 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:24.806022 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 203/404] 10.222.0.26 () {56 vars in 1180 bytes} [Sat Aug 6 01:20:24 2022] GET /identity/v3/groups/355b15c08fc7413fb84ea439cfe0c070 => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:24.812112 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a883cf-f152-4b89-8617-9dbb94858a20 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:24.812363 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 202/405] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Aug 6 01:20:24 2022] GET /identity/v3/projects/0bcfa6bfcd464fc9b0dc2079b24902bc => generated 302 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:24.813029 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a883cf-f152-4b89-8617-9dbb94858a20 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:24.813237 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0693c4bf-9ec8-4514-8f3b-42c92e1b9f69 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:24.813720 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a883cf-f152-4b89-8617-9dbb94858a20 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:24.815452 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 204/406] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:20:24 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:20:24.816417 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a883cf-f152-4b89-8617-9dbb94858a20 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:24.874967 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:24.918944 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0693c4bf-9ec8-4514-8f3b-42c92e1b9f69 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:24.931308 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a883cf-f152-4b89-8617-9dbb94858a20 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:24.954409 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1007376-bb8d-4217-9ae0-098c4c9e95af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:24.955115 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1007376-bb8d-4217-9ae0-098c4c9e95af None admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:24.955410 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1007376-bb8d-4217-9ae0-098c4c9e95af None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:24.955693 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1007376-bb8d-4217-9ae0-098c4c9e95af None admin] PATH_INFO: `/v3/projects/0bcfa6bfcd464fc9b0dc2079b24902bc/groups/355b15c08fc7413fb84ea439cfe0c070/roles/61b3b93d37a047e4bb8369b46fe20db9` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:24.957766 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1007376-bb8d-4217-9ae0-098c4c9e95af None admin] RBAC: Authorizing `identity:create_grant(project_id=0bcfa6bfcd464fc9b0dc2079b24902bc, group_id=355b15c08fc7413fb84ea439cfe0c070, role_id=61b3b93d37a047e4bb8369b46fe20db9)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:24.959835 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1007376-bb8d-4217-9ae0-098c4c9e95af None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:24.980916 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1d4f11-3300-44e5-9fdd-731c90549501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:24.985286 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1d4f11-3300-44e5-9fdd-731c90549501 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:24.985916 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1d4f11-3300-44e5-9fdd-731c90549501 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:24.986560 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1d4f11-3300-44e5-9fdd-731c90549501 None admin] PATH_INFO: `/v3/projects/0bcfa6bfcd464fc9b0dc2079b24902bc` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:24.986662 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 203/407] 10.222.0.26 () {58 vars in 1442 bytes} [Sat Aug 6 01:20:24 2022] PUT /identity/v3/projects/0bcfa6bfcd464fc9b0dc2079b24902bc/groups/355b15c08fc7413fb84ea439cfe0c070/roles/61b3b93d37a047e4bb8369b46fe20db9 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:20:24.988228 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ba0af9-6aa7-48ab-9848-d273e5ea18d3 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:24.988531 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ba0af9-6aa7-48ab-9848-d273e5ea18d3 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:24.989144 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ba0af9-6aa7-48ab-9848-d273e5ea18d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:24.990071 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1d4f11-3300-44e5-9fdd-731c90549501 None admin] RBAC: Authorizing `identity:get_project(project_id=0bcfa6bfcd464fc9b0dc2079b24902bc)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:24.991766 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1d4f11-3300-44e5-9fdd-731c90549501 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:24.995833 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 205/408] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Aug 6 01:20:24 2022] GET /identity/v3/projects/0bcfa6bfcd464fc9b0dc2079b24902bc => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:25.010383 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1d4f11-3300-44e5-9fdd-731c90549501 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:25.036974 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-15ba0af9-6aa7-48ab-9848-d273e5ea18d3 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:20:25.038309 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-15ba0af9-6aa7-48ab-9848-d273e5ea18d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:20:25.058661 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:25.154247 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1d4f11-3300-44e5-9fdd-731c90549501 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:25.172165 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 204/409] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:20:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:20:25.202590 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60600c56-657f-40ea-a6dc-60518b72e7ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:25.203379 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60600c56-657f-40ea-a6dc-60518b72e7ea None admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:25.203730 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60600c56-657f-40ea-a6dc-60518b72e7ea None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:25.204072 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60600c56-657f-40ea-a6dc-60518b72e7ea None admin] PATH_INFO: `/v3/projects/0bcfa6bfcd464fc9b0dc2079b24902bc/groups/355b15c08fc7413fb84ea439cfe0c070/roles/6d8b63d350944d78812edbefb32ae8ee` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:25.207295 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60600c56-657f-40ea-a6dc-60518b72e7ea None admin] RBAC: Authorizing `identity:create_grant(project_id=0bcfa6bfcd464fc9b0dc2079b24902bc, group_id=355b15c08fc7413fb84ea439cfe0c070, role_id=6d8b63d350944d78812edbefb32ae8ee)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:25.209098 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60600c56-657f-40ea-a6dc-60518b72e7ea None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:25.221310 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ba0af9-6aa7-48ab-9848-d273e5ea18d3 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:25.234125 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 206/410] 10.222.0.26 () {58 vars in 1442 bytes} [Sat Aug 6 01:20:25 2022] PUT /identity/v3/projects/0bcfa6bfcd464fc9b0dc2079b24902bc/groups/355b15c08fc7413fb84ea439cfe0c070/roles/6d8b63d350944d78812edbefb32ae8ee => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:20:25.330073 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:25.452457 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ba0af9-6aa7-48ab-9848-d273e5ea18d3 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:25.509359 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc25e32-1e67-444d-b1ba-714c9f5cb0f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:25.510019 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc25e32-1e67-444d-b1ba-714c9f5cb0f0 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:25.510304 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc25e32-1e67-444d-b1ba-714c9f5cb0f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:25.510582 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc25e32-1e67-444d-b1ba-714c9f5cb0f0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:25.536763 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc25e32-1e67-444d-b1ba-714c9f5cb0f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:20:25.548016 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-bfc25e32-1e67-444d-b1ba-714c9f5cb0f0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:20:25.548016 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:25.548016 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:25.548016 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:25.548016 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:25.548016 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:25.548016 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:25.548016 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:25.548016 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:25.548016 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:25.548016 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:25.548016 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:25.548016 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:20:25.548016 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:20:25.548016 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:20:25.548016 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:20:25.549341 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:20:25.549341 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:20:25.549341 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:20:25.549341 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:20:25.549341 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:25.549341 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:25.549341 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:25.549341 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:25.549341 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:25.549341 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:25.549341 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:25.549341 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:25.549341 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:25.549341 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:25.549341 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:25.549341 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:25.550213 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:25.550213 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:25.550213 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:25.550213 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:25.550213 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:20:25.550213 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:20:25.550213 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:20:25.550213 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:20:25.550213 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:20:25.550213 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:20:25.550213 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:20:25.550213 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:20:25.550213 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 205/411] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Aug 6 01:20:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:20:25.557166 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60600c56-657f-40ea-a6dc-60518b72e7ea None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:25.742774 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:25.765465 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60600c56-657f-40ea-a6dc-60518b72e7ea None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:25.810234 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebede31-841e-4eb5-8d0c-5e44b603b450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:25.810867 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebede31-841e-4eb5-8d0c-5e44b603b450 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:25.811152 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebede31-841e-4eb5-8d0c-5e44b603b450 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:25.811425 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebede31-841e-4eb5-8d0c-5e44b603b450 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:25.812156 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebede31-841e-4eb5-8d0c-5e44b603b450 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:25.812460 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebede31-841e-4eb5-8d0c-5e44b603b450 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:25.813855 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebede31-841e-4eb5-8d0c-5e44b603b450 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:25.829003 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 207/412] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Aug 6 01:20:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:25.834565 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc25e32-1e67-444d-b1ba-714c9f5cb0f0 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:25.864110 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbf0d49-f3b4-442d-866d-f490642b0ae8 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:25.864464 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbf0d49-f3b4-442d-866d-f490642b0ae8 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:25.864678 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbf0d49-f3b4-442d-866d-f490642b0ae8 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:25.867162 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 208/413] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:20:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:20:25.875124 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d719cdb-3f7d-47ab-abd6-592397c7b38b None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:25.875419 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d719cdb-3f7d-47ab-abd6-592397c7b38b None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:25.875693 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d719cdb-3f7d-47ab-abd6-592397c7b38b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:25.909312 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-9d719cdb-3f7d-47ab-abd6-592397c7b38b None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:20:25.910602 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-9d719cdb-3f7d-47ab-abd6-592397c7b38b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:20:25.972543 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc25e32-1e67-444d-b1ba-714c9f5cb0f0 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:25.982977 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:26.024838 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f833c14d-79d3-4b85-b0fd-c7cb91f67294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:26.025432 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f833c14d-79d3-4b85-b0fd-c7cb91f67294 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:26.025716 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f833c14d-79d3-4b85-b0fd-c7cb91f67294 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:26.025994 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f833c14d-79d3-4b85-b0fd-c7cb91f67294 None admin] PATH_INFO: `/v3/groups/f93e7761d988449a9be67baf7457c4cb` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:26.028612 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f833c14d-79d3-4b85-b0fd-c7cb91f67294 None admin] RBAC: Authorizing `identity:get_group(group_id=f93e7761d988449a9be67baf7457c4cb)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:26.030194 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f833c14d-79d3-4b85-b0fd-c7cb91f67294 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:26.031806 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 206/414] 10.222.0.26 () {56 vars in 1180 bytes} [Sat Aug 6 01:20:25 2022] GET /identity/v3/groups/f93e7761d988449a9be67baf7457c4cb => generated 229 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:26.037235 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f833c14d-79d3-4b85-b0fd-c7cb91f67294 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:26.056211 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 209/415] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:20:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:20:26.091252 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d719cdb-3f7d-47ab-abd6-592397c7b38b None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:26.201406 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:26.217439 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f833c14d-79d3-4b85-b0fd-c7cb91f67294 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:26.239959 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d719cdb-3f7d-47ab-abd6-592397c7b38b None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:26.282560 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7a8cdb-0908-4094-9031-c216f986a0ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:26.283565 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7a8cdb-0908-4094-9031-c216f986a0ac None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:26.284060 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7a8cdb-0908-4094-9031-c216f986a0ac None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:26.284549 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7a8cdb-0908-4094-9031-c216f986a0ac None admin] PATH_INFO: `/v3/projects/477e329599354f58af86e800a455a8b2` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:26.286529 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7a8cdb-0908-4094-9031-c216f986a0ac None admin] RBAC: Authorizing `identity:get_project(project_id=477e329599354f58af86e800a455a8b2)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:26.288179 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7a8cdb-0908-4094-9031-c216f986a0ac None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:26.292424 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 207/416] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Aug 6 01:20:26 2022] GET /identity/v3/projects/477e329599354f58af86e800a455a8b2 => generated 344 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:20:26.296001 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7a8cdb-0908-4094-9031-c216f986a0ac None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:26.300965 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6719fa38-920f-4048-b8d5-c1ffa1be49b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:26.301966 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6719fa38-920f-4048-b8d5-c1ffa1be49b6 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:26.303286 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6719fa38-920f-4048-b8d5-c1ffa1be49b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:26.303762 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6719fa38-920f-4048-b8d5-c1ffa1be49b6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:26.322531 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6719fa38-920f-4048-b8d5-c1ffa1be49b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:20:26.328939 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-6719fa38-920f-4048-b8d5-c1ffa1be49b6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 01:20:26.328939 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:26.328939 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:26.328939 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:26.328939 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:26.328939 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:26.328939 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:26.328939 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:26.328939 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:26.328939 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:26.328939 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:26.328939 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:26.328939 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:20:26.328939 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:20:26.328939 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:20:26.328939 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:20:26.329817 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:20:26.329817 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:20:26.329817 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:20:26.329817 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:20:26.329817 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:26.329817 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:26.329817 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:26.329817 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:26.329817 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:26.329817 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:26.329817 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:26.329817 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:26.329817 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:26.329817 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:26.329817 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:26.329817 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:26.330598 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:26.330598 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:26.330598 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:26.330598 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:26.330598 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:20:26.330598 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:20:26.330598 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:20:26.330598 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:20:26.330598 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:20:26.330598 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:20:26.330598 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 01:20:26.330598 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:20:26.337224 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 210/417] 10.222.0.26 () {56 vars in 1114 bytes} [Sat Aug 6 01:20:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:20:26.339291 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6719fa38-920f-4048-b8d5-c1ffa1be49b6 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:26.385556 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:26.463890 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7a8cdb-0908-4094-9031-c216f986a0ac None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:26.502934 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6ceae9-1a6a-492f-a826-fbf433b0b433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:26.503613 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6ceae9-1a6a-492f-a826-fbf433b0b433 None admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:26.503895 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6ceae9-1a6a-492f-a826-fbf433b0b433 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:26.504176 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6ceae9-1a6a-492f-a826-fbf433b0b433 None admin] PATH_INFO: `/v3/projects/477e329599354f58af86e800a455a8b2/groups/f93e7761d988449a9be67baf7457c4cb/roles/4559b9a0837249f895a13c70096f8d38` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:26.506803 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6ceae9-1a6a-492f-a826-fbf433b0b433 None admin] RBAC: Authorizing `identity:create_grant(project_id=477e329599354f58af86e800a455a8b2, group_id=f93e7761d988449a9be67baf7457c4cb, role_id=4559b9a0837249f895a13c70096f8d38)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:26.508402 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6ceae9-1a6a-492f-a826-fbf433b0b433 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:26.525661 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 208/418] 10.222.0.26 () {58 vars in 1442 bytes} [Sat Aug 6 01:20:26 2022] PUT /identity/v3/projects/477e329599354f58af86e800a455a8b2/groups/f93e7761d988449a9be67baf7457c4cb/roles/4559b9a0837249f895a13c70096f8d38 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:20:26.569602 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6719fa38-920f-4048-b8d5-c1ffa1be49b6 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:26.625755 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa99949-bb6e-4165-9a42-589c78dc139b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:26.626751 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa99949-bb6e-4165-9a42-589c78dc139b None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:26.627260 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa99949-bb6e-4165-9a42-589c78dc139b None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:26.627732 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa99949-bb6e-4165-9a42-589c78dc139b None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:26.628680 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa99949-bb6e-4165-9a42-589c78dc139b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:26.629235 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa99949-bb6e-4165-9a42-589c78dc139b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:26.630891 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa99949-bb6e-4165-9a42-589c78dc139b None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:26.639151 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:26.647196 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 211/419] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Aug 6 01:20:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:20:26.654250 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6ceae9-1a6a-492f-a826-fbf433b0b433 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:26.818382 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6ceae9-1a6a-492f-a826-fbf433b0b433 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:26.822125 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:26.860225 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f258c08a-dd83-4248-8082-e10f37ea42b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:26.861259 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f258c08a-dd83-4248-8082-e10f37ea42b4 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:26.861742 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f258c08a-dd83-4248-8082-e10f37ea42b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:26.862231 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f258c08a-dd83-4248-8082-e10f37ea42b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:26.943214 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f258c08a-dd83-4248-8082-e10f37ea42b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:26.944996 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f258c08a-dd83-4248-8082-e10f37ea42b4 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:27.007181 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c8b202-3ae9-4302-8742-c94102e2ee59 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:27.008015 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c8b202-3ae9-4302-8742-c94102e2ee59 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:27.008437 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c8b202-3ae9-4302-8742-c94102e2ee59 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:27.009970 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 212/420] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:20:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:20:27.018981 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b40f93-9846-4e9c-a7e4-366106dd3fae None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:27.019269 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b40f93-9846-4e9c-a7e4-366106dd3fae None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:27.019546 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b40f93-9846-4e9c-a7e4-366106dd3fae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:27.049618 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-63b40f93-9846-4e9c-a7e4-366106dd3fae None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:20:27.050803 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-63b40f93-9846-4e9c-a7e4-366106dd3fae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:20:27.070364 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 209/421] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Aug 6 01:20:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 01:20:27.076575 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f258c08a-dd83-4248-8082-e10f37ea42b4 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:27.079528 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:27.174364 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 213/422] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:20:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:20:27.205936 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f258c08a-dd83-4248-8082-e10f37ea42b4 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:27.210615 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b40f93-9846-4e9c-a7e4-366106dd3fae None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:27.243678 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4803e9bc-d7d6-44da-bb05-ab6212e03b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:27.244333 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4803e9bc-d7d6-44da-bb05-ab6212e03b47 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:27.244618 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4803e9bc-d7d6-44da-bb05-ab6212e03b47 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:27.244899 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4803e9bc-d7d6-44da-bb05-ab6212e03b47 None admin] PATH_INFO: `/v3/projects/730685c4f62b43c493d42b5897c99a02` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:27.246115 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4803e9bc-d7d6-44da-bb05-ab6212e03b47 None admin] RBAC: Authorizing `identity:get_project(project_id=730685c4f62b43c493d42b5897c99a02)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:27.248311 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4803e9bc-d7d6-44da-bb05-ab6212e03b47 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:27.250555 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 210/423] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Aug 6 01:20:27 2022] GET /identity/v3/projects/730685c4f62b43c493d42b5897c99a02 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:20:27.256632 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4803e9bc-d7d6-44da-bb05-ab6212e03b47 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:27.349266 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:27.379687 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b40f93-9846-4e9c-a7e4-366106dd3fae None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:27.423670 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:27.433420 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a8486c-9e42-46bd-a936-d9fb47d562de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:27.434768 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a8486c-9e42-46bd-a936-d9fb47d562de None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:27.435261 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a8486c-9e42-46bd-a936-d9fb47d562de None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:27.435726 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a8486c-9e42-46bd-a936-d9fb47d562de None admin] PATH_INFO: `/v3/roles/member` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:27.447527 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a8486c-9e42-46bd-a936-d9fb47d562de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:20:27.448268 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4803e9bc-d7d6-44da-bb05-ab6212e03b47 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:27.453639 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-61a8486c-9e42-46bd-a936-d9fb47d562de None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 06 01:20:27.453639 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:27.453639 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:27.453639 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:27.453639 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:27.453639 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:27.453639 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:27.453639 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:27.453639 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:27.453639 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:27.453639 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:27.453639 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:27.453639 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:20:27.453639 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:20:27.453639 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:20:27.453639 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:20:27.454508 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:20:27.454508 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:20:27.454508 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:20:27.454508 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:20:27.454508 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:27.454508 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:27.454508 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:27.454508 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:27.454508 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:27.454508 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:27.454508 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:27.454508 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:27.454508 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:27.454508 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:27.454508 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:27.454508 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:27.455304 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:27.455304 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:27.455304 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:27.455304 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:27.455304 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:20:27.455304 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:20:27.455304 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:20:27.455304 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:20:27.455304 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:20:27.455304 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:20:27.455304 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 06 01:20:27.455304 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:20:27.462435 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 214/424] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Aug 6 01:20:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:20:27.474013 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a8486c-9e42-46bd-a936-d9fb47d562de None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:27.493653 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fc54b0-a04e-413a-9c02-b9b439e4e0b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:27.494628 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fc54b0-a04e-413a-9c02-b9b439e4e0b1 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:27.495463 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fc54b0-a04e-413a-9c02-b9b439e4e0b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:27.495936 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fc54b0-a04e-413a-9c02-b9b439e4e0b1 None admin] PATH_INFO: `/v3/groups/355b15c08fc7413fb84ea439cfe0c070` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:27.497858 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fc54b0-a04e-413a-9c02-b9b439e4e0b1 None admin] RBAC: Authorizing `identity:get_group(group_id=355b15c08fc7413fb84ea439cfe0c070)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:27.499500 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fc54b0-a04e-413a-9c02-b9b439e4e0b1 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:27.501660 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 211/425] 10.222.0.26 () {56 vars in 1180 bytes} [Sat Aug 6 01:20:27 2022] GET /identity/v3/groups/355b15c08fc7413fb84ea439cfe0c070 => generated 226 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:20:27.507810 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fc54b0-a04e-413a-9c02-b9b439e4e0b1 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:27.615879 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fc54b0-a04e-413a-9c02-b9b439e4e0b1 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:27.633291 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a8486c-9e42-46bd-a936-d9fb47d562de None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:27.651892 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6e6d4d-6f7b-4a1c-92c8-c150d0f2ea55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:27.653482 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6e6d4d-6f7b-4a1c-92c8-c150d0f2ea55 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:27.654044 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6e6d4d-6f7b-4a1c-92c8-c150d0f2ea55 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:27.654510 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6e6d4d-6f7b-4a1c-92c8-c150d0f2ea55 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:27.655930 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6e6d4d-6f7b-4a1c-92c8-c150d0f2ea55 None admin] RBAC: Adding query filter params (group.id=355b15c08fc7413fb84ea439cfe0c070, role.id=6d8b63d350944d78812edbefb32ae8ee, scope.project.id=730685c4f62b43c493d42b5897c99a02) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:27.656440 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6e6d4d-6f7b-4a1c-92c8-c150d0f2ea55 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:27.658077 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6e6d4d-6f7b-4a1c-92c8-c150d0f2ea55 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:27.664387 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 212/426] 10.222.0.26 () {56 vars in 1376 bytes} [Sat Aug 6 01:20:27 2022] GET /identity/v3/role_assignments?group.id=355b15c08fc7413fb84ea439cfe0c070&scope.project.id=730685c4f62b43c493d42b5897c99a02&role.id=6d8b63d350944d78812edbefb32ae8ee => generated 611 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:27.670065 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4f9da5-3119-4a06-a3f3-9edb601f261a None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:27.670356 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4f9da5-3119-4a06-a3f3-9edb601f261a None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:27.670635 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4f9da5-3119-4a06-a3f3-9edb601f261a None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:27.672037 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 213/427] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:20:27 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:20:27.678943 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e04488e-d9df-4333-a73b-f38dad7ad2d8 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:27.679235 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e04488e-d9df-4333-a73b-f38dad7ad2d8 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:27.679509 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e04488e-d9df-4333-a73b-f38dad7ad2d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:27.684491 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de256563-f231-490c-9b46-889f18954b9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:27.685316 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de256563-f231-490c-9b46-889f18954b9b None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:27.685601 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de256563-f231-490c-9b46-889f18954b9b None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:27.685879 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de256563-f231-490c-9b46-889f18954b9b None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:27.686627 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de256563-f231-490c-9b46-889f18954b9b None admin] RBAC: Adding query filter params (name=member) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:27.686938 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de256563-f231-490c-9b46-889f18954b9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:27.688358 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de256563-f231-490c-9b46-889f18954b9b None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:27.702445 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:27.708406 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 215/428] 10.222.0.26 () {56 vars in 1101 bytes} [Sat Aug 6 01:20:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:27.718125 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de256563-f231-490c-9b46-889f18954b9b None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:27.733447 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-5e04488e-d9df-4333-a73b-f38dad7ad2d8 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:20:27.734701 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-5e04488e-d9df-4333-a73b-f38dad7ad2d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:20:27.818913 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:27.866071 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 214/429] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:20:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:20:27.867098 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1210d16c-9d47-4560-97f4-082598005b06 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:27.867487 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1210d16c-9d47-4560-97f4-082598005b06 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:27.867774 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1210d16c-9d47-4560-97f4-082598005b06 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:27.869193 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 215/430] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:20:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:20:27.874942 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb7d226-178a-4c0d-aa39-f907c2779e96 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:27.875227 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb7d226-178a-4c0d-aa39-f907c2779e96 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:27.875616 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb7d226-178a-4c0d-aa39-f907c2779e96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:27.890214 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de256563-f231-490c-9b46-889f18954b9b None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:27.895574 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-5fb7d226-178a-4c0d-aa39-f907c2779e96 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:20:27.896765 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-5fb7d226-178a-4c0d-aa39-f907c2779e96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:20:27.938895 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712b1745-aef6-4a00-8fdb-4bc8be35d577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:27.938895 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712b1745-aef6-4a00-8fdb-4bc8be35d577 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:27.939159 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712b1745-aef6-4a00-8fdb-4bc8be35d577 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:27.939365 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712b1745-aef6-4a00-8fdb-4bc8be35d577 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:28.001648 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 216/431] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:20:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:20:28.001648 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb7d226-178a-4c0d-aa39-f907c2779e96 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:28.025576 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712b1745-aef6-4a00-8fdb-4bc8be35d577 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:28.027206 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712b1745-aef6-4a00-8fdb-4bc8be35d577 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:28.062234 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:28.083409 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:28.110033 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb7d226-178a-4c0d-aa39-f907c2779e96 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:28.141340 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 216/432] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Aug 6 01:20:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 01:20:28.142203 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712b1745-aef6-4a00-8fdb-4bc8be35d577 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:28.150463 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff1611b-b058-4187-8f0c-b845e93bb71b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:28.151115 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff1611b-b058-4187-8f0c-b845e93bb71b None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:28.151399 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff1611b-b058-4187-8f0c-b845e93bb71b None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:28.151678 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff1611b-b058-4187-8f0c-b845e93bb71b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:28.162777 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff1611b-b058-4187-8f0c-b845e93bb71b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:20:28.167802 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-3ff1611b-b058-4187-8f0c-b845e93bb71b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 06 01:20:28.167802 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:28.167802 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:28.167802 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:28.167802 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:28.167802 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:28.167802 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:28.167802 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:28.167802 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:28.167802 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:28.167802 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:28.167802 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:28.167802 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:20:28.167802 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:20:28.167802 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:20:28.167802 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:20:28.168823 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:20:28.168823 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:20:28.168823 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:20:28.168823 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:20:28.168823 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:28.168823 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:28.168823 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:28.168823 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:28.168823 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:28.168823 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:28.168823 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:28.168823 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:28.168823 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:28.168823 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:28.168823 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:28.168823 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:28.169639 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:28.169639 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:28.169639 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:28.169639 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:28.169639 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:20:28.169639 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:20:28.169639 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:20:28.169639 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:20:28.169639 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:20:28.169639 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:20:28.169639 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 06 01:20:28.169639 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:20:28.173999 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 217/433] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Aug 6 01:20:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:20:28.174821 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff1611b-b058-4187-8f0c-b845e93bb71b None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:28.249938 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712b1745-aef6-4a00-8fdb-4bc8be35d577 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:28.284221 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff1611b-b058-4187-8f0c-b845e93bb71b None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:28.287235 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339e91d0-7d47-42ac-aa9a-18f219ced1c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:28.287872 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339e91d0-7d47-42ac-aa9a-18f219ced1c8 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:28.288154 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339e91d0-7d47-42ac-aa9a-18f219ced1c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:28.288430 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339e91d0-7d47-42ac-aa9a-18f219ced1c8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:28.299742 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339e91d0-7d47-42ac-aa9a-18f219ced1c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:20:28.304608 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-339e91d0-7d47-42ac-aa9a-18f219ced1c8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 01:20:28.304608 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:28.304608 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:28.304608 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:28.304608 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:28.304608 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:28.304608 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:28.304608 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:28.304608 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:28.304608 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:28.304608 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:28.304608 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:28.304608 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:20:28.304608 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:20:28.304608 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:20:28.304608 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:20:28.305552 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:20:28.305552 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:20:28.305552 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:20:28.305552 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:20:28.305552 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:28.305552 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:28.305552 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:28.305552 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:28.305552 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:28.305552 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:28.305552 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:28.305552 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:28.305552 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:28.305552 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:28.305552 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:28.305552 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:28.306343 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:28.306343 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:28.306343 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:28.306343 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:28.306343 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:20:28.306343 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:20:28.306343 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:20:28.306343 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:20:28.306343 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:20:28.306343 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:20:28.306343 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 01:20:28.306343 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:20:28.306343 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 217/434] 10.222.0.26 () {56 vars in 1114 bytes} [Sat Aug 6 01:20:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:20:28.307032 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339e91d0-7d47-42ac-aa9a-18f219ced1c8 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:28.312759 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:28.322413 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:28.325673 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff908bfa-6478-4ee8-b07c-885ad72b2f99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:28.326315 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff908bfa-6478-4ee8-b07c-885ad72b2f99 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:28.326600 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff908bfa-6478-4ee8-b07c-885ad72b2f99 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:28.326878 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff908bfa-6478-4ee8-b07c-885ad72b2f99 None admin] PATH_INFO: `/v3/projects/730685c4f62b43c493d42b5897c99a02` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:28.328125 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff908bfa-6478-4ee8-b07c-885ad72b2f99 None admin] RBAC: Authorizing `identity:get_project(project_id=730685c4f62b43c493d42b5897c99a02)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:28.329811 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff908bfa-6478-4ee8-b07c-885ad72b2f99 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:28.331681 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 218/435] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Aug 6 01:20:28 2022] GET /identity/v3/projects/730685c4f62b43c493d42b5897c99a02 => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:28.332336 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff908bfa-6478-4ee8-b07c-885ad72b2f99 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:28.424038 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339e91d0-7d47-42ac-aa9a-18f219ced1c8 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:28.447098 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff908bfa-6478-4ee8-b07c-885ad72b2f99 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:28.460203 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dca8ce-2100-42fa-83ef-a4e8fb03b947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:28.460825 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dca8ce-2100-42fa-83ef-a4e8fb03b947 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:28.461133 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dca8ce-2100-42fa-83ef-a4e8fb03b947 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:28.461414 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dca8ce-2100-42fa-83ef-a4e8fb03b947 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:28.462134 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dca8ce-2100-42fa-83ef-a4e8fb03b947 None admin] RBAC: Adding query filter params (name=member) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:28.462441 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dca8ce-2100-42fa-83ef-a4e8fb03b947 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:28.463797 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dca8ce-2100-42fa-83ef-a4e8fb03b947 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:28.472361 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 218/436] 10.222.0.26 () {56 vars in 1101 bytes} [Sat Aug 6 01:20:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:28.473238 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dca8ce-2100-42fa-83ef-a4e8fb03b947 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:28.491081 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3d24f7-0411-4fb8-9907-3730d70e44b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:28.491718 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3d24f7-0411-4fb8-9907-3730d70e44b0 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:28.492008 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3d24f7-0411-4fb8-9907-3730d70e44b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:28.492291 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3d24f7-0411-4fb8-9907-3730d70e44b0 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:28.493009 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3d24f7-0411-4fb8-9907-3730d70e44b0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:28.493352 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3d24f7-0411-4fb8-9907-3730d70e44b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:28.494716 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3d24f7-0411-4fb8-9907-3730d70e44b0 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:28.507547 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 219/437] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Aug 6 01:20:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:28.507547 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3d24f7-0411-4fb8-9907-3730d70e44b0 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:28.557032 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:28.566018 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:28.589631 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dca8ce-2100-42fa-83ef-a4e8fb03b947 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:28.614257 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3d24f7-0411-4fb8-9907-3730d70e44b0 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:28.625866 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bda4736-f4c6-4c90-9283-96f59d84eb0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:28.626509 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bda4736-f4c6-4c90-9283-96f59d84eb0a None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:28.626796 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bda4736-f4c6-4c90-9283-96f59d84eb0a None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:28.627075 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bda4736-f4c6-4c90-9283-96f59d84eb0a None admin] PATH_INFO: `/v3/groups/355b15c08fc7413fb84ea439cfe0c070` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:28.628309 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bda4736-f4c6-4c90-9283-96f59d84eb0a None admin] RBAC: Authorizing `identity:get_group(group_id=355b15c08fc7413fb84ea439cfe0c070)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:28.629776 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bda4736-f4c6-4c90-9283-96f59d84eb0a None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:28.631381 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 219/438] 10.222.0.26 () {56 vars in 1180 bytes} [Sat Aug 6 01:20:28 2022] GET /identity/v3/groups/355b15c08fc7413fb84ea439cfe0c070 => generated 226 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:28.632139 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bda4736-f4c6-4c90-9283-96f59d84eb0a None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:28.656915 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c3a782-2b7f-4f4e-bf84-fb64632d3bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:28.657487 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c3a782-2b7f-4f4e-bf84-fb64632d3bca None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:28.657724 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c3a782-2b7f-4f4e-bf84-fb64632d3bca None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:28.658004 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c3a782-2b7f-4f4e-bf84-fb64632d3bca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:28.731590 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c3a782-2b7f-4f4e-bf84-fb64632d3bca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:28.733041 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c3a782-2b7f-4f4e-bf84-fb64632d3bca None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:28.739947 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bda4736-f4c6-4c90-9283-96f59d84eb0a None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:28.774077 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4abd02-21f8-4257-a58d-775a9c06df34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:28.774695 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4abd02-21f8-4257-a58d-775a9c06df34 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:28.774984 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4abd02-21f8-4257-a58d-775a9c06df34 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:28.775258 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4abd02-21f8-4257-a58d-775a9c06df34 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:28.786827 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:28.789194 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:28.844462 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 220/439] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Aug 6 01:20:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 01:20:28.845266 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c3a782-2b7f-4f4e-bf84-fb64632d3bca None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:28.848069 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4abd02-21f8-4257-a58d-775a9c06df34 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:28.849650 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4abd02-21f8-4257-a58d-775a9c06df34 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:28.951481 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c3a782-2b7f-4f4e-bf84-fb64632d3bca None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:28.955865 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 220/440] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Aug 6 01:20:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 01:20:28.957273 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4abd02-21f8-4257-a58d-775a9c06df34 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:28.989849 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04912258-8009-40a7-bcd0-ea3255dccdfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:28.990472 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04912258-8009-40a7-bcd0-ea3255dccdfb None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:28.990759 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04912258-8009-40a7-bcd0-ea3255dccdfb None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:28.991051 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04912258-8009-40a7-bcd0-ea3255dccdfb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:28.991852 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04912258-8009-40a7-bcd0-ea3255dccdfb None admin] RBAC: Adding query filter params (group.id=355b15c08fc7413fb84ea439cfe0c070, role.id=61b3b93d37a047e4bb8369b46fe20db9, scope.project.id=730685c4f62b43c493d42b5897c99a02) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:28.992168 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04912258-8009-40a7-bcd0-ea3255dccdfb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:28.993643 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04912258-8009-40a7-bcd0-ea3255dccdfb None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:28.999807 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:29.013084 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 221/441] 10.222.0.26 () {56 vars in 1376 bytes} [Sat Aug 6 01:20:28 2022] GET /identity/v3/role_assignments?group.id=355b15c08fc7413fb84ea439cfe0c070&scope.project.id=730685c4f62b43c493d42b5897c99a02&role.id=61b3b93d37a047e4bb8369b46fe20db9 => generated 611 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:29.013084 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04912258-8009-40a7-bcd0-ea3255dccdfb None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:29.024322 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:29.072682 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4abd02-21f8-4257-a58d-775a9c06df34 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:29.112760 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9450dcd1-00f7-4f89-878a-b504b1c31658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:29.113482 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9450dcd1-00f7-4f89-878a-b504b1c31658 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:29.113767 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9450dcd1-00f7-4f89-878a-b504b1c31658 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:29.114049 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9450dcd1-00f7-4f89-878a-b504b1c31658 None admin] PATH_INFO: `/v3/projects/0bcfa6bfcd464fc9b0dc2079b24902bc` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:29.115291 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9450dcd1-00f7-4f89-878a-b504b1c31658 None admin] RBAC: Authorizing `identity:get_project(project_id=0bcfa6bfcd464fc9b0dc2079b24902bc)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:29.116764 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9450dcd1-00f7-4f89-878a-b504b1c31658 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:29.118532 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 221/442] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Aug 6 01:20:28 2022] GET /identity/v3/projects/0bcfa6bfcd464fc9b0dc2079b24902bc => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:29.119743 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b68e22-9497-4eb4-ace9-972ad97cfa7a None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:29.120029 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b68e22-9497-4eb4-ace9-972ad97cfa7a None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:29.120308 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b68e22-9497-4eb4-ace9-972ad97cfa7a None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:29.126667 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 222/443] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:20:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:20:29.126667 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b68e22-9497-4eb4-ace9-972ad97cfa7a None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:29.136907 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04912258-8009-40a7-bcd0-ea3255dccdfb None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:29.184648 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c0b0b0-96c1-42eb-8751-266782d62d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:29.185494 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c0b0b0-96c1-42eb-8751-266782d62d71 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:29.185783 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c0b0b0-96c1-42eb-8751-266782d62d71 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:29.186087 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c0b0b0-96c1-42eb-8751-266782d62d71 None admin] PATH_INFO: `/v3/projects/0bcfa6bfcd464fc9b0dc2079b24902bc` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:29.187296 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c0b0b0-96c1-42eb-8751-266782d62d71 None admin] RBAC: Authorizing `identity:get_project(project_id=0bcfa6bfcd464fc9b0dc2079b24902bc)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:29.188751 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c0b0b0-96c1-42eb-8751-266782d62d71 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:29.190467 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 222/444] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Aug 6 01:20:29 2022] GET /identity/v3/projects/0bcfa6bfcd464fc9b0dc2079b24902bc => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:29.191607 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9640ccfa-c9ae-4341-a69a-49d2efa3408f None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:29.191938 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9640ccfa-c9ae-4341-a69a-49d2efa3408f None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:29.192167 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9640ccfa-c9ae-4341-a69a-49d2efa3408f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:29.216422 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-9640ccfa-c9ae-4341-a69a-49d2efa3408f None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:20:29.217598 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-9640ccfa-c9ae-4341-a69a-49d2efa3408f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:20:29.255489 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b68e22-9497-4eb4-ace9-972ad97cfa7a None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:29.267391 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:29.299259 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5fa17a-e060-475b-8f10-bd059f84046f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:29.301292 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5fa17a-e060-475b-8f10-bd059f84046f None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:29.301292 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5fa17a-e060-475b-8f10-bd059f84046f None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:29.301292 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5fa17a-e060-475b-8f10-bd059f84046f None admin] PATH_INFO: `/v3/groups/355b15c08fc7413fb84ea439cfe0c070` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:29.302195 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5fa17a-e060-475b-8f10-bd059f84046f None admin] RBAC: Authorizing `identity:get_group(group_id=355b15c08fc7413fb84ea439cfe0c070)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:29.302296 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 223/445] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:20:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:20:29.303076 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9640ccfa-c9ae-4341-a69a-49d2efa3408f None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:29.303511 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5fa17a-e060-475b-8f10-bd059f84046f None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:29.305615 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 223/446] 10.222.0.26 () {56 vars in 1180 bytes} [Sat Aug 6 01:20:29 2022] GET /identity/v3/groups/355b15c08fc7413fb84ea439cfe0c070 => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:29.311146 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5fa17a-e060-475b-8f10-bd059f84046f None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:29.330393 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:29.429890 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9640ccfa-c9ae-4341-a69a-49d2efa3408f None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:29.431254 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5fa17a-e060-475b-8f10-bd059f84046f None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:29.472596 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270034e9-939a-4c7c-b97b-143cba96f53a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:29.473121 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f504e36-b3dd-49d2-ba46-87b5aa127c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:29.473264 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270034e9-939a-4c7c-b97b-143cba96f53a None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:29.473494 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270034e9-939a-4c7c-b97b-143cba96f53a None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:29.473742 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f504e36-b3dd-49d2-ba46-87b5aa127c14 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:29.473852 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270034e9-939a-4c7c-b97b-143cba96f53a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:29.474025 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f504e36-b3dd-49d2-ba46-87b5aa127c14 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:29.474302 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f504e36-b3dd-49d2-ba46-87b5aa127c14 None admin] PATH_INFO: `/v3/groups/355b15c08fc7413fb84ea439cfe0c070` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:29.474549 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270034e9-939a-4c7c-b97b-143cba96f53a None admin] RBAC: Adding query filter params (group.id=355b15c08fc7413fb84ea439cfe0c070, role.id=61b3b93d37a047e4bb8369b46fe20db9, scope.project.id=0bcfa6bfcd464fc9b0dc2079b24902bc) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:29.474857 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270034e9-939a-4c7c-b97b-143cba96f53a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:29.475464 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f504e36-b3dd-49d2-ba46-87b5aa127c14 None admin] RBAC: Authorizing `identity:get_group(group_id=355b15c08fc7413fb84ea439cfe0c070)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:29.476555 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270034e9-939a-4c7c-b97b-143cba96f53a None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:29.477183 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f504e36-b3dd-49d2-ba46-87b5aa127c14 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:29.483455 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 224/447] 10.222.0.26 () {56 vars in 1180 bytes} [Sat Aug 6 01:20:29 2022] GET /identity/v3/groups/355b15c08fc7413fb84ea439cfe0c070 => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:29.483455 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f504e36-b3dd-49d2-ba46-87b5aa127c14 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:29.483694 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 224/448] 10.222.0.26 () {56 vars in 1376 bytes} [Sat Aug 6 01:20:29 2022] GET /identity/v3/role_assignments?group.id=355b15c08fc7413fb84ea439cfe0c070&scope.project.id=0bcfa6bfcd464fc9b0dc2079b24902bc&role.id=61b3b93d37a047e4bb8369b46fe20db9 => generated 611 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:20:29.490703 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270034e9-939a-4c7c-b97b-143cba96f53a None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:29.549149 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:29.584683 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:29.609926 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f504e36-b3dd-49d2-ba46-87b5aa127c14 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:29.629432 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270034e9-939a-4c7c-b97b-143cba96f53a None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:29.653097 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89edb4d-5f51-40a9-b1c6-0f3b6d84ddc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:29.653664 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89edb4d-5f51-40a9-b1c6-0f3b6d84ddc7 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:29.654103 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89edb4d-5f51-40a9-b1c6-0f3b6d84ddc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:29.654489 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89edb4d-5f51-40a9-b1c6-0f3b6d84ddc7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:29.666104 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89edb4d-5f51-40a9-b1c6-0f3b6d84ddc7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:20:29.671764 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-d89edb4d-5f51-40a9-b1c6-0f3b6d84ddc7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:20:29.671764 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:29.671764 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:29.671764 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:29.671764 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:29.671764 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:29.671764 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:29.671764 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:29.671764 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:29.671764 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:29.671764 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:29.671764 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:29.671764 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:20:29.671764 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:20:29.671764 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:20:29.671764 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:20:29.672649 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:20:29.672649 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:20:29.672649 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:20:29.672649 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:20:29.672649 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:29.672649 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:29.672649 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:29.672649 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:29.672649 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:29.672649 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:29.672649 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:29.672649 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:29.672649 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:29.672649 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:29.672649 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:29.672649 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:29.673494 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:29.673494 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:29.673494 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:29.673494 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:29.673494 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:20:29.673494 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:20:29.673494 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:20:29.673494 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:20:29.673494 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:20:29.673494 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:20:29.673494 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:20:29.673494 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:20:29.673494 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 225/449] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Aug 6 01:20:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:20:29.679114 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89edb4d-5f51-40a9-b1c6-0f3b6d84ddc7 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:29.681706 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526086a0-40bd-4d2a-b869-67a31c1ff88d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:29.681873 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526086a0-40bd-4d2a-b869-67a31c1ff88d None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:29.682075 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526086a0-40bd-4d2a-b869-67a31c1ff88d None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:29.682312 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526086a0-40bd-4d2a-b869-67a31c1ff88d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:29.683111 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526086a0-40bd-4d2a-b869-67a31c1ff88d None admin] RBAC: Adding query filter params (group.id=355b15c08fc7413fb84ea439cfe0c070, role.id=6d8b63d350944d78812edbefb32ae8ee, scope.project.id=0bcfa6bfcd464fc9b0dc2079b24902bc) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:29.683424 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526086a0-40bd-4d2a-b869-67a31c1ff88d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:29.685538 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526086a0-40bd-4d2a-b869-67a31c1ff88d None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:29.692088 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 225/450] 10.222.0.26 () {56 vars in 1376 bytes} [Sat Aug 6 01:20:29 2022] GET /identity/v3/role_assignments?group.id=355b15c08fc7413fb84ea439cfe0c070&scope.project.id=0bcfa6bfcd464fc9b0dc2079b24902bc&role.id=6d8b63d350944d78812edbefb32ae8ee => generated 611 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:20:29.801197 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89edb4d-5f51-40a9-b1c6-0f3b6d84ddc7 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:29.839462 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aee87c-089f-4516-8495-e42eac2f03e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:29.840136 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aee87c-089f-4516-8495-e42eac2f03e5 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:29.840416 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aee87c-089f-4516-8495-e42eac2f03e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:29.840697 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aee87c-089f-4516-8495-e42eac2f03e5 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:29.841530 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aee87c-089f-4516-8495-e42eac2f03e5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:29.841836 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aee87c-089f-4516-8495-e42eac2f03e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:29.843211 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aee87c-089f-4516-8495-e42eac2f03e5 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:29.852651 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 226/451] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Aug 6 01:20:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:29.859101 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526086a0-40bd-4d2a-b869-67a31c1ff88d None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:29.962824 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526086a0-40bd-4d2a-b869-67a31c1ff88d None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:29.966323 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:29.999298 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57580c2d-62d9-4c7d-a651-65c63676e2a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:29.999916 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57580c2d-62d9-4c7d-a651-65c63676e2a8 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:30.000198 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57580c2d-62d9-4c7d-a651-65c63676e2a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:30.000475 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57580c2d-62d9-4c7d-a651-65c63676e2a8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:30.067607 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57580c2d-62d9-4c7d-a651-65c63676e2a8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:30.069019 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57580c2d-62d9-4c7d-a651-65c63676e2a8 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:30.164436 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 226/452] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Aug 6 01:20:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 01:20:30.170343 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aee87c-089f-4516-8495-e42eac2f03e5 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:30.179466 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:30.271054 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aee87c-089f-4516-8495-e42eac2f03e5 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:30.302607 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed937ce-b93c-4254-ac7e-a8cdd10969f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:30.303240 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed937ce-b93c-4254-ac7e-a8cdd10969f3 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:30.303523 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed937ce-b93c-4254-ac7e-a8cdd10969f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:30.303797 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed937ce-b93c-4254-ac7e-a8cdd10969f3 None admin] PATH_INFO: `/v3/projects/477e329599354f58af86e800a455a8b2` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:30.304954 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed937ce-b93c-4254-ac7e-a8cdd10969f3 None admin] RBAC: Authorizing `identity:get_project(project_id=477e329599354f58af86e800a455a8b2)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:30.306405 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed937ce-b93c-4254-ac7e-a8cdd10969f3 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:30.308049 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 227/453] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Aug 6 01:20:30 2022] GET /identity/v3/projects/477e329599354f58af86e800a455a8b2 => generated 344 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:30.314268 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57580c2d-62d9-4c7d-a651-65c63676e2a8 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:30.317302 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:30.409950 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57580c2d-62d9-4c7d-a651-65c63676e2a8 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:30.443337 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077cdf61-ac21-4abd-b98a-8411e3b7ed28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:30.443964 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077cdf61-ac21-4abd-b98a-8411e3b7ed28 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:30.444246 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077cdf61-ac21-4abd-b98a-8411e3b7ed28 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:30.444521 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077cdf61-ac21-4abd-b98a-8411e3b7ed28 None admin] PATH_INFO: `/v3/groups/f93e7761d988449a9be67baf7457c4cb` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:30.445652 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077cdf61-ac21-4abd-b98a-8411e3b7ed28 None admin] RBAC: Authorizing `identity:get_group(group_id=f93e7761d988449a9be67baf7457c4cb)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:30.447078 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077cdf61-ac21-4abd-b98a-8411e3b7ed28 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:30.448653 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 227/454] 10.222.0.26 () {56 vars in 1180 bytes} [Sat Aug 6 01:20:30 2022] GET /identity/v3/groups/f93e7761d988449a9be67baf7457c4cb => generated 229 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:30.455296 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed937ce-b93c-4254-ac7e-a8cdd10969f3 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:30.529757 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:30.553474 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed937ce-b93c-4254-ac7e-a8cdd10969f3 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:30.586423 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4951e8ac-f302-4d8f-a7b2-3497083e2a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:30.587026 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4951e8ac-f302-4d8f-a7b2-3497083e2a58 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:30.587308 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4951e8ac-f302-4d8f-a7b2-3497083e2a58 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:30.587585 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4951e8ac-f302-4d8f-a7b2-3497083e2a58 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:30.588366 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4951e8ac-f302-4d8f-a7b2-3497083e2a58 None admin] RBAC: Adding query filter params (group.id=f93e7761d988449a9be67baf7457c4cb, role.id=4559b9a0837249f895a13c70096f8d38, scope.project.id=477e329599354f58af86e800a455a8b2) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:30.588676 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4951e8ac-f302-4d8f-a7b2-3497083e2a58 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:30.590358 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4951e8ac-f302-4d8f-a7b2-3497083e2a58 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:30.596044 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 228/455] 10.222.0.26 () {56 vars in 1376 bytes} [Sat Aug 6 01:20:30 2022] GET /identity/v3/role_assignments?group.id=f93e7761d988449a9be67baf7457c4cb&scope.project.id=477e329599354f58af86e800a455a8b2&role.id=4559b9a0837249f895a13c70096f8d38 => generated 611 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:32.986825 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee802bba-7794-4c38-96d0-16c807217b63 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:32.987338 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee802bba-7794-4c38-96d0-16c807217b63 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:32.987449 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee802bba-7794-4c38-96d0-16c807217b63 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:32.989140 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 228/456] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:20:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:20:33.002236 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace8e16a-0817-40a7-94bb-90ac38a37f99 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:33.002654 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace8e16a-0817-40a7-94bb-90ac38a37f99 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:33.002794 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace8e16a-0817-40a7-94bb-90ac38a37f99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:33.024820 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-ace8e16a-0817-40a7-94bb-90ac38a37f99 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:20:33.026059 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-ace8e16a-0817-40a7-94bb-90ac38a37f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:20:33.115204 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 229/457] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:20:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:20:33.124004 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a86291a-425e-43f6-a8b5-dbd586c5f6f9 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:33.124298 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a86291a-425e-43f6-a8b5-dbd586c5f6f9 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:33.124580 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a86291a-425e-43f6-a8b5-dbd586c5f6f9 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:33.126072 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 229/458] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:20:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:20:33.138214 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e428fad4-5a20-46a1-9372-f1ec76e9f144 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:33.138507 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e428fad4-5a20-46a1-9372-f1ec76e9f144 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:33.138792 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e428fad4-5a20-46a1-9372-f1ec76e9f144 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:33.147255 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a86291a-425e-43f6-a8b5-dbd586c5f6f9 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:33.166902 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-e428fad4-5a20-46a1-9372-f1ec76e9f144 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:20:33.168646 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-e428fad4-5a20-46a1-9372-f1ec76e9f144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:20:33.266532 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:33.268960 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 230/459] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:20:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:20:33.302330 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e428fad4-5a20-46a1-9372-f1ec76e9f144 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:33.306853 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:33.355181 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a86291a-425e-43f6-a8b5-dbd586c5f6f9 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:33.406896 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6c90ab-fead-451c-acb0-d0b058b0abff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:33.407546 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6c90ab-fead-451c-acb0-d0b058b0abff None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:33.407831 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6c90ab-fead-451c-acb0-d0b058b0abff None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:33.408110 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6c90ab-fead-451c-acb0-d0b058b0abff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:33.425383 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6c90ab-fead-451c-acb0-d0b058b0abff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:33.427508 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6c90ab-fead-451c-acb0-d0b058b0abff None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:33.448563 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-fe6c90ab-fead-451c-acb0-d0b058b0abff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:20:33.448563 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:33.448563 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:33.448563 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:33.448563 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:33.448563 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:33.448563 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:33.448563 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:33.448563 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:33.448563 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:33.448563 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:33.448563 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:33.448563 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:20:33.448563 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:20:33.448563 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:20:33.450135 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:20:33.450135 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:33.450135 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:33.450135 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:33.450135 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:33.450135 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:33.450135 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:33.450135 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:33.450135 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:33.450135 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:33.450135 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:33.450135 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:33.450135 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:33.450135 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:33.450135 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:33.450135 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:33.450135 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:33.450950 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:20:33.450950 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:20:33.450950 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:20:33.450950 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:20:33.452506 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 230/460] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:20:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:20:33.453855 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e428fad4-5a20-46a1-9372-f1ec76e9f144 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:33.454034 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9e26d2-ccfc-475f-9d88-3b20e52f4ce2 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:33.454356 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9e26d2-ccfc-475f-9d88-3b20e52f4ce2 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:33.454690 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9e26d2-ccfc-475f-9d88-3b20e52f4ce2 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:33.457450 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 231/461] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:20:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:20:33.462438 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9e26d2-ccfc-475f-9d88-3b20e52f4ce2 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:33.496503 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f820bfef-e7d5-4d4f-ae60-fea980164e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:33.497592 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f820bfef-e7d5-4d4f-ae60-fea980164e2d None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:33.498017 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f820bfef-e7d5-4d4f-ae60-fea980164e2d None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:33.498425 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f820bfef-e7d5-4d4f-ae60-fea980164e2d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:33.512512 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f820bfef-e7d5-4d4f-ae60-fea980164e2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:33.514088 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f820bfef-e7d5-4d4f-ae60-fea980164e2d None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:33.534773 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-f820bfef-e7d5-4d4f-ae60-fea980164e2d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:20:33.534773 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:33.534773 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:33.534773 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:33.534773 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:33.534773 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:33.534773 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:33.534773 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:33.534773 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:33.534773 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:33.534773 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:33.534773 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:33.534773 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:20:33.534773 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:20:33.534773 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:20:33.536401 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:20:33.536401 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:33.536401 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:33.536401 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:33.536401 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:33.536401 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:33.536401 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:33.536401 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:33.536401 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:33.536401 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:33.536401 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:33.536401 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:33.536401 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:33.536401 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:33.536401 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:33.536401 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:33.536401 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:33.537563 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:20:33.537563 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:20:33.537563 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:20:33.537563 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:20:33.541343 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 231/462] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:20:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:20:33.543486 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8995c5c-419a-45c7-bbc7-7c4919f00014 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:33.543792 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8995c5c-419a-45c7-bbc7-7c4919f00014 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:33.544077 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8995c5c-419a-45c7-bbc7-7c4919f00014 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:33.565996 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-a8995c5c-419a-45c7-bbc7-7c4919f00014 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:20:33.567645 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-a8995c5c-419a-45c7-bbc7-7c4919f00014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:20:33.602626 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9e26d2-ccfc-475f-9d88-3b20e52f4ce2 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:33.624484 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:33.634804 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:33.650951 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 232/463] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:20:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:20:33.651960 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8995c5c-419a-45c7-bbc7-7c4919f00014 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:33.652880 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8a1d8c-7c57-4f09-875e-a083e7d2c277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:33.653707 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8a1d8c-7c57-4f09-875e-a083e7d2c277 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:33.653992 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8a1d8c-7c57-4f09-875e-a083e7d2c277 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:33.654271 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8a1d8c-7c57-4f09-875e-a083e7d2c277 None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:33.657328 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8a1d8c-7c57-4f09-875e-a083e7d2c277 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:33.657660 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8a1d8c-7c57-4f09-875e-a083e7d2c277 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:33.659062 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8a1d8c-7c57-4f09-875e-a083e7d2c277 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:33.685395 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 232/464] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Aug 6 01:20:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:33.686318 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8a1d8c-7c57-4f09-875e-a083e7d2c277 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:33.795243 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8a1d8c-7c57-4f09-875e-a083e7d2c277 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:33.801163 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8995c5c-419a-45c7-bbc7-7c4919f00014 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:33.835057 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580575ba-c6a6-42f1-97d4-1a37a35a51ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:33.836019 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580575ba-c6a6-42f1-97d4-1a37a35a51ab None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:33.836306 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580575ba-c6a6-42f1-97d4-1a37a35a51ab None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:33.836832 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580575ba-c6a6-42f1-97d4-1a37a35a51ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:33.843282 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700b6a7a-6447-403a-8b86-261ff4aaa974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:33.843906 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700b6a7a-6447-403a-8b86-261ff4aaa974 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:33.844187 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700b6a7a-6447-403a-8b86-261ff4aaa974 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:33.844459 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700b6a7a-6447-403a-8b86-261ff4aaa974 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:33.845189 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700b6a7a-6447-403a-8b86-261ff4aaa974 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:33.845493 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700b6a7a-6447-403a-8b86-261ff4aaa974 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:33.846868 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700b6a7a-6447-403a-8b86-261ff4aaa974 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:33.848601 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580575ba-c6a6-42f1-97d4-1a37a35a51ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:33.850596 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580575ba-c6a6-42f1-97d4-1a37a35a51ab None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:33.860152 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 233/465] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Aug 6 01:20:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:33.861121 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700b6a7a-6447-403a-8b86-261ff4aaa974 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:33.872070 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-580575ba-c6a6-42f1-97d4-1a37a35a51ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:20:33.872070 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:33.872070 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:33.872070 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:33.872070 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:33.872070 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:33.872070 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:33.872070 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:33.872070 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:33.872070 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:33.872070 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:33.872070 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:33.872070 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:20:33.872070 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:20:33.872070 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:20:33.874342 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:20:33.874342 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:33.874342 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:33.874342 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:33.874342 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:33.874342 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:33.874342 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:33.874342 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:33.874342 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:33.874342 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:33.874342 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:33.874342 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:33.874342 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:33.874342 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:33.874342 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:33.874342 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:33.874342 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:33.875526 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:20:33.875526 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:20:33.875526 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:20:33.875526 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:20:33.875526 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 233/466] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:20:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:20:33.875526 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580575ba-c6a6-42f1-97d4-1a37a35a51ab None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:33.914994 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:33.916878 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:33.985853 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700b6a7a-6447-403a-8b86-261ff4aaa974 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:33.986358 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580575ba-c6a6-42f1-97d4-1a37a35a51ab None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:34.022611 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fb6092-2128-4bcf-a5e6-7954d5dca4df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:34.023390 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fb6092-2128-4bcf-a5e6-7954d5dca4df None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:34.023735 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fb6092-2128-4bcf-a5e6-7954d5dca4df None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:34.023964 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fb6092-2128-4bcf-a5e6-7954d5dca4df None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:34.024389 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e92ad9-7dc9-422c-8c36-818fd505ba56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:34.024756 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fb6092-2128-4bcf-a5e6-7954d5dca4df None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:34.025064 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e92ad9-7dc9-422c-8c36-818fd505ba56 None admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:34.025352 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e92ad9-7dc9-422c-8c36-818fd505ba56 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:34.025625 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e92ad9-7dc9-422c-8c36-818fd505ba56 None admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:34.026114 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fb6092-2128-4bcf-a5e6-7954d5dca4df None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:34.026300 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e92ad9-7dc9-422c-8c36-818fd505ba56 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:34.028254 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e92ad9-7dc9-422c-8c36-818fd505ba56 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:34.048985 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 234/467] 10.222.0.26 () {60 vars in 1132 bytes} [Sat Aug 6 01:20:33 2022] POST /identity/v3/users => generated 246 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:20:34.050157 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fb6092-2128-4bcf-a5e6-7954d5dca4df None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:34.061327 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 234/468] 10.222.0.26 () {60 vars in 1132 bytes} [Sat Aug 6 01:20:33 2022] POST /identity/v3/users => generated 248 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:20:34.063076 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348f1e4f-0fd6-440f-a14f-b9672cdd58b4 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:34.063582 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348f1e4f-0fd6-440f-a14f-b9672cdd58b4 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:34.064046 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348f1e4f-0fd6-440f-a14f-b9672cdd58b4 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:34.066053 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 235/469] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:20:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:20:34.072100 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e84e16c-48d7-4198-ad93-1e16264dd61b None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:34.072740 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e84e16c-48d7-4198-ad93-1e16264dd61b None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:34.073268 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e84e16c-48d7-4198-ad93-1e16264dd61b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:34.126264 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-4e84e16c-48d7-4198-ad93-1e16264dd61b None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:20:34.127846 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-4e84e16c-48d7-4198-ad93-1e16264dd61b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:20:34.207623 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:34.224897 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fb6092-2128-4bcf-a5e6-7954d5dca4df None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:34.272082 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 236/470] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:20:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:20:34.286354 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60a95b5-b45b-41c2-8879-ddce1456f380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:34.287075 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60a95b5-b45b-41c2-8879-ddce1456f380 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:34.287432 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60a95b5-b45b-41c2-8879-ddce1456f380 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:34.287785 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60a95b5-b45b-41c2-8879-ddce1456f380 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:34.288592 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60a95b5-b45b-41c2-8879-ddce1456f380 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:34.288968 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60a95b5-b45b-41c2-8879-ddce1456f380 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:34.290557 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60a95b5-b45b-41c2-8879-ddce1456f380 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:34.303009 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e84e16c-48d7-4198-ad93-1e16264dd61b None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:34.305700 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 235/471] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Aug 6 01:20:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:34.317374 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60a95b5-b45b-41c2-8879-ddce1456f380 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:34.335995 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:34.458248 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e84e16c-48d7-4198-ad93-1e16264dd61b None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:34.473243 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60a95b5-b45b-41c2-8879-ddce1456f380 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:34.509486 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8f4d6b-4e14-44bd-87af-9d97d5b25f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:34.510535 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8f4d6b-4e14-44bd-87af-9d97d5b25f06 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:34.510966 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8f4d6b-4e14-44bd-87af-9d97d5b25f06 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:34.511389 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8f4d6b-4e14-44bd-87af-9d97d5b25f06 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:34.513147 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0ae633-d703-4b35-8a7a-bedbd4f4db46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:34.513781 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0ae633-d703-4b35-8a7a-bedbd4f4db46 None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:34.514123 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0ae633-d703-4b35-8a7a-bedbd4f4db46 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:34.514418 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0ae633-d703-4b35-8a7a-bedbd4f4db46 None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:34.515102 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0ae633-d703-4b35-8a7a-bedbd4f4db46 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:34.516560 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0ae633-d703-4b35-8a7a-bedbd4f4db46 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:34.523823 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8f4d6b-4e14-44bd-87af-9d97d5b25f06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:34.526212 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8f4d6b-4e14-44bd-87af-9d97d5b25f06 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:34.545331 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-0d8f4d6b-4e14-44bd-87af-9d97d5b25f06 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:20:34.545331 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:34.545331 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:34.545331 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:34.545331 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:34.545331 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:34.545331 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:34.545331 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:34.545331 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:34.545331 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:34.545331 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:34.545331 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:34.545331 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:20:34.545331 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:20:34.545331 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:20:34.546283 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:20:34.546283 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:34.546283 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:34.546283 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:34.546283 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:34.546283 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:34.546283 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:34.546283 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:34.546283 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:34.546283 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:34.546283 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:34.546283 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:34.546283 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:34.546283 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:34.546283 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:34.546283 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:34.546283 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:34.547132 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:20:34.547132 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:20:34.547132 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:20:34.547132 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:20:34.549251 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 236/472] 10.222.0.26 () {60 vars in 1132 bytes} [Sat Aug 6 01:20:34 2022] POST /identity/v3/users => generated 248 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:20:34.549667 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 237/473] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:20:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:20:34.562932 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0ae633-d703-4b35-8a7a-bedbd4f4db46 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:34.587508 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:34.618320 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0335c5-8f6c-4816-9894-aeb272b0d765 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:34.618709 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0335c5-8f6c-4816-9894-aeb272b0d765 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:34.619074 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0335c5-8f6c-4816-9894-aeb272b0d765 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:34.620825 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 238/474] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:20:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 01:20:34.628701 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072b4135-1eee-414f-98dc-3dd92410bd5d None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:34.629081 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072b4135-1eee-414f-98dc-3dd92410bd5d None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:34.629603 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072b4135-1eee-414f-98dc-3dd92410bd5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:34.663648 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-072b4135-1eee-414f-98dc-3dd92410bd5d None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:20:34.664883 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-072b4135-1eee-414f-98dc-3dd92410bd5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:20:34.758671 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 239/475] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:20:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:20:34.765661 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0ae633-d703-4b35-8a7a-bedbd4f4db46 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:34.796286 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072b4135-1eee-414f-98dc-3dd92410bd5d None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:34.801377 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:34.807409 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9df5d2-465a-4615-855f-490302368761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:34.807920 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9df5d2-465a-4615-855f-490302368761 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:34.808300 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9df5d2-465a-4615-855f-490302368761 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:34.808710 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9df5d2-465a-4615-855f-490302368761 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:34.809537 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9df5d2-465a-4615-855f-490302368761 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:34.809853 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9df5d2-465a-4615-855f-490302368761 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:34.811227 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9df5d2-465a-4615-855f-490302368761 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:34.824729 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 237/476] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Aug 6 01:20:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:34.829730 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9df5d2-465a-4615-855f-490302368761 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:34.922561 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072b4135-1eee-414f-98dc-3dd92410bd5d None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:34.923571 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:34.959162 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9df5d2-465a-4615-855f-490302368761 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:34.965991 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b46213f-5b33-42d7-b397-72d2c966d0eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:34.966625 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b46213f-5b33-42d7-b397-72d2c966d0eb None admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:34.966910 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b46213f-5b33-42d7-b397-72d2c966d0eb None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:34.967192 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b46213f-5b33-42d7-b397-72d2c966d0eb None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:34.967917 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b46213f-5b33-42d7-b397-72d2c966d0eb None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:34.969814 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b46213f-5b33-42d7-b397-72d2c966d0eb None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:34.985126 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.sql.core [None req-2b46213f-5b33-42d7-b397-72d2c966d0eb None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 06 01:20:34.985126 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 06 01:20:34.985126 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [parameters: {'id': 'e83d390f0e1a4798b1df61c1282b5c6e', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 06 01:20:34.985126 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80202) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 06 01:20:35.001504 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-2b46213f-5b33-42d7-b397-72d2c966d0eb None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 06 01:20:35.001504 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:35.001504 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 06 01:20:35.001504 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 06 01:20:35.001504 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 06 01:20:35.001504 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 06 01:20:35.001504 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 06 01:20:35.001504 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application result = self._query(query) Aug 06 01:20:35.001504 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 06 01:20:35.001504 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application conn.query(q) Aug 06 01:20:35.001504 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 06 01:20:35.001504 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 06 01:20:35.001504 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 06 01:20:35.001504 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application result.read() Aug 06 01:20:35.001504 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 06 01:20:35.002456 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 06 01:20:35.002456 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 06 01:20:35.002456 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 06 01:20:35.002456 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 06 01:20:35.002456 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 06 01:20:35.002456 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 06 01:20:35.002456 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 06 01:20:35.002456 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 06 01:20:35.002456 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:20:35.002456 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 06 01:20:35.002456 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:20:35.002456 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:35.002456 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 06 01:20:35.002456 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 06 01:20:35.002456 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 06 01:20:35.002456 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return ref.to_dict() Aug 06 01:20:35.002456 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 06 01:20:35.002456 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application next(self.gen) Aug 06 01:20:35.003413 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 06 01:20:35.003413 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application yield resource Aug 06 01:20:35.003413 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 06 01:20:35.003413 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application next(self.gen) Aug 06 01:20:35.003413 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 06 01:20:35.003413 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self.session.rollback() Aug 06 01:20:35.003413 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 06 01:20:35.003413 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self.force_reraise() Aug 06 01:20:35.003413 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 06 01:20:35.003413 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise self.value Aug 06 01:20:35.003413 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 06 01:20:35.003413 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 06 01:20:35.003413 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 06 01:20:35.003413 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application session.commit() Aug 06 01:20:35.003413 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 06 01:20:35.003413 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 06 01:20:35.003413 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 06 01:20:35.004268 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self._prepare_impl() Aug 06 01:20:35.004268 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 06 01:20:35.004268 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self.session.flush() Aug 06 01:20:35.004268 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 06 01:20:35.004268 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self._flush(objects) Aug 06 01:20:35.004268 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 06 01:20:35.004268 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 06 01:20:35.004268 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 06 01:20:35.004268 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application compat.raise_( Aug 06 01:20:35.004268 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 06 01:20:35.004268 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception Aug 06 01:20:35.004268 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 06 01:20:35.004268 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application flush_context.execute() Aug 06 01:20:35.004268 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 06 01:20:35.004268 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rec.execute(self) Aug 06 01:20:35.004268 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 06 01:20:35.004268 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 06 01:20:35.005978 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 06 01:20:35.005978 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 06 01:20:35.005978 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 06 01:20:35.005978 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 06 01:20:35.005978 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 06 01:20:35.005978 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 06 01:20:35.005978 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 06 01:20:35.005978 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 06 01:20:35.005978 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 06 01:20:35.005978 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 06 01:20:35.005978 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 06 01:20:35.005978 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 06 01:20:35.005978 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 06 01:20:35.005978 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 06 01:20:35.005978 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 06 01:20:35.005978 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception Aug 06 01:20:35.007812 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 06 01:20:35.007812 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 06 01:20:35.007812 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 06 01:20:35.007812 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 06 01:20:35.007812 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 06 01:20:35.007812 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application result = self._query(query) Aug 06 01:20:35.007812 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 06 01:20:35.007812 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application conn.query(q) Aug 06 01:20:35.007812 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 06 01:20:35.007812 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 06 01:20:35.007812 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 06 01:20:35.007812 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application result.read() Aug 06 01:20:35.007812 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 06 01:20:35.007812 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 06 01:20:35.007812 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 06 01:20:35.007812 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 06 01:20:35.009925 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 06 01:20:35.009925 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 06 01:20:35.009925 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 06 01:20:35.009925 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 06 01:20:35.009925 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 06 01:20:35.009925 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 06 01:20:35.009925 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [parameters: {'id': 'e83d390f0e1a4798b1df61c1282b5c6e', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 06 01:20:35.009925 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 06 01:20:35.009925 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:20:35.009925 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 01:20:35.009925 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:20:35.009925 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:35.009925 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:35.009925 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:35.009925 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:35.009925 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:35.017858 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:35.017858 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:35.017858 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:35.017858 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:35.017858 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:35.017858 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:35.017858 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 06 01:20:35.017858 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 06 01:20:35.017858 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:35.017858 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:35.017858 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 06 01:20:35.017858 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 06 01:20:35.017858 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 06 01:20:35.017858 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 06 01:20:35.017858 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 06 01:20:35.017858 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:20:35.020006 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 240/477] 10.222.0.26 () {60 vars in 1131 bytes} [Sat Aug 6 01:20:34 2022] POST /identity/v3/roles => generated 144 bytes in 209 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:20:35.020006 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b46213f-5b33-42d7-b397-72d2c966d0eb None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:35.048344 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9926d8f2-df9c-4f8f-9fd3-144603e752f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:35.049737 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9926d8f2-df9c-4f8f-9fd3-144603e752f2 None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:35.050222 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9926d8f2-df9c-4f8f-9fd3-144603e752f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:35.050696 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9926d8f2-df9c-4f8f-9fd3-144603e752f2 None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:35.051604 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9926d8f2-df9c-4f8f-9fd3-144603e752f2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:35.056189 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9926d8f2-df9c-4f8f-9fd3-144603e752f2 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:35.089516 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:35.102902 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 238/478] 10.222.0.26 () {60 vars in 1132 bytes} [Sat Aug 6 01:20:34 2022] POST /identity/v3/users => generated 249 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:20:35.131842 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b46213f-5b33-42d7-b397-72d2c966d0eb None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:35.171937 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35643a31-3e1f-4244-881c-5cb69b300310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:35.172663 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35643a31-3e1f-4244-881c-5cb69b300310 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:35.172957 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35643a31-3e1f-4244-881c-5cb69b300310 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:35.173346 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35643a31-3e1f-4244-881c-5cb69b300310 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:35.186099 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35643a31-3e1f-4244-881c-5cb69b300310 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:20:35.191368 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-35643a31-3e1f-4244-881c-5cb69b300310 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 01:20:35.191368 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:35.191368 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:35.191368 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:35.191368 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:35.191368 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:35.191368 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:35.191368 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:35.191368 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:35.191368 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:35.191368 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:35.191368 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:35.191368 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:20:35.191368 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:20:35.191368 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:20:35.191368 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:20:35.192266 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:20:35.192266 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:20:35.192266 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:20:35.192266 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:20:35.192266 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:35.192266 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:35.192266 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:35.192266 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:35.192266 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:35.192266 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:35.192266 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:35.192266 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:35.192266 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:35.192266 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:35.192266 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:35.192266 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:35.193093 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:35.193093 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:35.193093 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:35.193093 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:35.193093 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:20:35.193093 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:20:35.193093 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:20:35.193093 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:20:35.193093 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:20:35.193093 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:20:35.193093 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 01:20:35.193093 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:20:35.196433 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 241/479] 10.222.0.26 () {56 vars in 1114 bytes} [Sat Aug 6 01:20:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:20:35.199754 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9926d8f2-df9c-4f8f-9fd3-144603e752f2 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:35.312589 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9926d8f2-df9c-4f8f-9fd3-144603e752f2 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:35.342403 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:35.351408 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b13a92-0578-4bc8-ad5c-9e21113a112e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:35.352054 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b13a92-0578-4bc8-ad5c-9e21113a112e None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:35.352378 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b13a92-0578-4bc8-ad5c-9e21113a112e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:35.352622 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b13a92-0578-4bc8-ad5c-9e21113a112e None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:35.353583 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b13a92-0578-4bc8-ad5c-9e21113a112e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:35.353961 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b13a92-0578-4bc8-ad5c-9e21113a112e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:35.356023 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b13a92-0578-4bc8-ad5c-9e21113a112e None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:35.369208 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 239/480] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Aug 6 01:20:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:36.558771 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f63a90-a863-41e3-993e-bdcae41478c0 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:36.559304 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f63a90-a863-41e3-993e-bdcae41478c0 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:36.559564 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f63a90-a863-41e3-993e-bdcae41478c0 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:36.561066 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 242/481] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:20:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:20:36.566228 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33958c8c-8bac-475b-bfb7-bf67efc92ef7 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:36.566621 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33958c8c-8bac-475b-bfb7-bf67efc92ef7 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:36.566786 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33958c8c-8bac-475b-bfb7-bf67efc92ef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:36.592133 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-33958c8c-8bac-475b-bfb7-bf67efc92ef7 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:20:36.593375 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-33958c8c-8bac-475b-bfb7-bf67efc92ef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:20:36.672928 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 240/482] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:20:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:20:36.695653 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f63a90-a863-41e3-993e-bdcae41478c0 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:36.821164 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f63a90-a863-41e3-993e-bdcae41478c0 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:36.849623 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:36.858550 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcefc10-faba-4ca0-98c2-7839c695974c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:36.859198 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcefc10-faba-4ca0-98c2-7839c695974c None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:36.859481 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcefc10-faba-4ca0-98c2-7839c695974c None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:36.859756 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcefc10-faba-4ca0-98c2-7839c695974c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:36.871880 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f25fad2-34f2-438a-b327-c345a5e843d6 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:36.872187 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f25fad2-34f2-438a-b327-c345a5e843d6 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:36.872440 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f25fad2-34f2-438a-b327-c345a5e843d6 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:36.873815 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 241/483] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:20:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:20:36.874695 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcefc10-faba-4ca0-98c2-7839c695974c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:20:36.878816 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937d7b43-968a-482f-a646-1a825dda6c9f None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:36.879303 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937d7b43-968a-482f-a646-1a825dda6c9f None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:36.879645 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-fbcefc10-faba-4ca0-98c2-7839c695974c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 06 01:20:36.879645 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:36.879645 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:36.879645 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:36.879645 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:36.879645 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:36.879645 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:36.879645 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:36.879645 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:36.879645 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:36.879645 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:36.879645 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:36.879645 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:20:36.879645 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:20:36.879645 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:20:36.879645 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:20:36.880488 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:20:36.880488 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:20:36.880488 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:20:36.880488 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:20:36.880488 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:36.880488 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:36.880488 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:36.880488 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:36.880488 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:36.880488 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:36.880488 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:36.880488 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:36.880488 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:36.880488 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:36.880488 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:36.880488 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:36.881302 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:36.881302 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:36.881302 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:36.881302 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:36.881302 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:20:36.881302 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:20:36.881302 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:20:36.881302 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:20:36.881302 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:20:36.881302 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:20:36.881302 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 06 01:20:36.881302 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:20:36.881302 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 243/484] 10.222.0.26 () {56 vars in 1102 bytes} [Sat Aug 6 01:20:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:20:36.882570 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937d7b43-968a-482f-a646-1a825dda6c9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:36.893067 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcefc10-faba-4ca0-98c2-7839c695974c None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:36.926118 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-937d7b43-968a-482f-a646-1a825dda6c9f None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:20:36.927600 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-937d7b43-968a-482f-a646-1a825dda6c9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:20:36.998541 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcefc10-faba-4ca0-98c2-7839c695974c None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:37.036084 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c114eeee-9a00-4950-af6d-ad5fb29be87d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:37.036731 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c114eeee-9a00-4950-af6d-ad5fb29be87d None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:37.037019 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c114eeee-9a00-4950-af6d-ad5fb29be87d None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:37.037364 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c114eeee-9a00-4950-af6d-ad5fb29be87d None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:37.038091 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c114eeee-9a00-4950-af6d-ad5fb29be87d None admin] RBAC: Adding query filter params (name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:37.038397 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c114eeee-9a00-4950-af6d-ad5fb29be87d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:37.039772 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c114eeee-9a00-4950-af6d-ad5fb29be87d None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:37.049484 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 244/485] 10.222.0.26 () {56 vars in 1103 bytes} [Sat Aug 6 01:20:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:20:37.054948 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c114eeee-9a00-4950-af6d-ad5fb29be87d None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:37.110492 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:37.110709 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 242/486] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:20:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:20:37.147651 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937d7b43-968a-482f-a646-1a825dda6c9f None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:37.166604 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c114eeee-9a00-4950-af6d-ad5fb29be87d None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:37.203123 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bce91d3-fc10-440c-a9aa-1853e7fba067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:37.204304 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bce91d3-fc10-440c-a9aa-1853e7fba067 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:37.204863 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bce91d3-fc10-440c-a9aa-1853e7fba067 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:37.205359 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bce91d3-fc10-440c-a9aa-1853e7fba067 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:37.218632 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bce91d3-fc10-440c-a9aa-1853e7fba067 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:37.220418 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bce91d3-fc10-440c-a9aa-1853e7fba067 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:37.237720 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-4bce91d3-fc10-440c-a9aa-1853e7fba067 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:20:37.237720 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:37.237720 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:37.237720 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:37.237720 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:37.237720 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:37.237720 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:37.237720 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:37.237720 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:37.237720 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:37.237720 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:37.237720 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:37.237720 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:20:37.237720 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:20:37.237720 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:20:37.238616 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:20:37.238616 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:37.238616 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:37.238616 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:37.238616 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:37.238616 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:37.238616 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:37.238616 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:37.238616 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:37.238616 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:37.238616 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:37.238616 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:37.238616 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:37.238616 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:37.238616 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:37.238616 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:37.238616 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:37.239467 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:20:37.239467 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:20:37.239467 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:20:37.239467 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:20:37.242602 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 245/487] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:20:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:20:37.249276 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bce91d3-fc10-440c-a9aa-1853e7fba067 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:37.282426 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:37.357680 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937d7b43-968a-482f-a646-1a825dda6c9f None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:37.376748 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bce91d3-fc10-440c-a9aa-1853e7fba067 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:37.379016 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:37.398498 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c84c595-3411-4b36-ac78-6936cc29da56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:37.399240 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c84c595-3411-4b36-ac78-6936cc29da56 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:37.399733 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c84c595-3411-4b36-ac78-6936cc29da56 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:37.400074 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c84c595-3411-4b36-ac78-6936cc29da56 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:37.420175 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c84c595-3411-4b36-ac78-6936cc29da56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:20:37.421113 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c91154c-3427-42bd-8142-16a4e87dcf84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:37.421937 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c91154c-3427-42bd-8142-16a4e87dcf84 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:37.422425 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c91154c-3427-42bd-8142-16a4e87dcf84 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:37.422877 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c91154c-3427-42bd-8142-16a4e87dcf84 None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:37.423745 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c91154c-3427-42bd-8142-16a4e87dcf84 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:37.424236 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c91154c-3427-42bd-8142-16a4e87dcf84 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:37.425798 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c91154c-3427-42bd-8142-16a4e87dcf84 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:37.425903 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-0c84c595-3411-4b36-ac78-6936cc29da56 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 06 01:20:37.425903 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:37.425903 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:37.425903 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:37.425903 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:37.425903 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:37.425903 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:37.425903 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:37.425903 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:37.425903 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:37.425903 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:37.425903 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:37.425903 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:20:37.425903 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:20:37.425903 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:20:37.425903 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:20:37.426726 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:20:37.426726 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:20:37.426726 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:20:37.426726 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:20:37.426726 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:37.426726 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:37.426726 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:37.426726 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:37.426726 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:37.426726 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:37.426726 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:37.426726 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:37.426726 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:37.426726 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:37.426726 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:37.426726 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:37.427496 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:37.427496 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:37.427496 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:37.427496 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:37.427496 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:20:37.427496 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:20:37.427496 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:20:37.427496 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:20:37.427496 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:20:37.427496 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:20:37.427496 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 06 01:20:37.427496 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:20:37.427496 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 243/488] 10.222.0.26 () {56 vars in 1102 bytes} [Sat Aug 6 01:20:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:20:37.429203 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eee48a-d184-42b4-bc2e-8bd387cc853e None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:37.429548 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eee48a-d184-42b4-bc2e-8bd387cc853e None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:37.429875 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eee48a-d184-42b4-bc2e-8bd387cc853e None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:37.433121 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 244/489] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:20:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:20:37.434654 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eee48a-d184-42b4-bc2e-8bd387cc853e None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:37.452024 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 246/490] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Aug 6 01:20:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:20:37.457024 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd1b5cb-6bf3-4f47-ab52-dc74fa4585b2 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:37.457024 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd1b5cb-6bf3-4f47-ab52-dc74fa4585b2 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:37.457024 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd1b5cb-6bf3-4f47-ab52-dc74fa4585b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:37.479245 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-8dd1b5cb-6bf3-4f47-ab52-dc74fa4585b2 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:20:37.481513 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-8dd1b5cb-6bf3-4f47-ab52-dc74fa4585b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:20:37.561588 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 247/491] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:20:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:20:37.563258 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd1b5cb-6bf3-4f47-ab52-dc74fa4585b2 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:37.586141 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eee48a-d184-42b4-bc2e-8bd387cc853e None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:37.598535 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:37.628561 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01466e12-69ab-4664-ba08-e59b80b5757d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:37.629332 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01466e12-69ab-4664-ba08-e59b80b5757d None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:37.630053 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01466e12-69ab-4664-ba08-e59b80b5757d None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:37.630402 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01466e12-69ab-4664-ba08-e59b80b5757d None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:37.631237 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01466e12-69ab-4664-ba08-e59b80b5757d None admin] RBAC: Adding query filter params (name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:37.631606 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01466e12-69ab-4664-ba08-e59b80b5757d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:37.633101 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01466e12-69ab-4664-ba08-e59b80b5757d None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:37.644997 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 245/492] 10.222.0.26 () {56 vars in 1103 bytes} [Sat Aug 6 01:20:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:37.646003 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01466e12-69ab-4664-ba08-e59b80b5757d None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:37.682309 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:37.695101 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd1b5cb-6bf3-4f47-ab52-dc74fa4585b2 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:37.739761 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353537d5-c049-488d-8a9f-68e5f97eec1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:37.740801 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353537d5-c049-488d-8a9f-68e5f97eec1b None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:37.741333 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353537d5-c049-488d-8a9f-68e5f97eec1b None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:37.741783 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353537d5-c049-488d-8a9f-68e5f97eec1b None admin] PATH_INFO: `/v3/users/glance` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:37.753077 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353537d5-c049-488d-8a9f-68e5f97eec1b None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:37.755296 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353537d5-c049-488d-8a9f-68e5f97eec1b None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:37.772539 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01466e12-69ab-4664-ba08-e59b80b5757d None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:37.773811 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-353537d5-c049-488d-8a9f-68e5f97eec1b None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 06 01:20:37.773811 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:37.773811 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:37.773811 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:37.773811 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:37.773811 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:37.773811 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:37.773811 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:37.773811 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:37.773811 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:37.773811 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:37.773811 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:37.773811 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 01:20:37.773811 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 01:20:37.773811 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 01:20:37.773811 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 01:20:37.774686 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:37.774686 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:37.774686 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 01:20:37.774686 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:20:37.774686 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 01:20:37.774686 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:20:37.774686 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:37.774686 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:37.774686 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:37.774686 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:37.774686 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:37.774686 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:37.774686 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:37.774686 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:37.774686 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:37.774686 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:37.774686 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:37.774686 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:37.775580 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:37.775580 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:37.775580 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 01:20:37.775580 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 01:20:37.775580 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 01:20:37.775580 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 01:20:37.775580 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 01:20:37.775580 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 01:20:37.775580 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 06 01:20:37.775580 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:20:37.779106 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 248/493] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Aug 6 01:20:37 2022] GET /identity/v3/users/glance => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:20:37.780045 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353537d5-c049-488d-8a9f-68e5f97eec1b None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:37.820624 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9b45e9-a671-4449-9f53-95b98331b937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:37.821307 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9b45e9-a671-4449-9f53-95b98331b937 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:37.821590 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9b45e9-a671-4449-9f53-95b98331b937 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:37.822044 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9b45e9-a671-4449-9f53-95b98331b937 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:37.833780 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9b45e9-a671-4449-9f53-95b98331b937 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:20:37.839169 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-5f9b45e9-a671-4449-9f53-95b98331b937 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 06 01:20:37.839169 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:37.839169 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:37.839169 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:37.839169 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:37.839169 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:37.839169 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:37.839169 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:37.839169 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:37.839169 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:37.839169 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:37.839169 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:37.839169 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:20:37.839169 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:20:37.839169 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:20:37.839169 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:20:37.840103 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:20:37.840103 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:20:37.840103 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:20:37.840103 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:20:37.840103 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:37.840103 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:37.840103 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:37.840103 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:37.840103 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:37.840103 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:37.840103 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:37.840103 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:37.840103 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:37.840103 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:37.840103 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:37.840103 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:37.846398 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:37.846398 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:37.846398 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:37.846398 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:37.846398 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:20:37.846398 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:20:37.846398 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:20:37.846398 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:20:37.846398 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:20:37.846398 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:20:37.846398 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 06 01:20:37.846398 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:20:37.846398 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 246/494] 10.222.0.26 () {56 vars in 1102 bytes} [Sat Aug 6 01:20:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:20:37.846398 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9b45e9-a671-4449-9f53-95b98331b937 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:37.879387 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:37.911574 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353537d5-c049-488d-8a9f-68e5f97eec1b None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:37.945361 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d77b829-e1bd-4624-ba89-5eb91577b3f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:37.946034 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d77b829-e1bd-4624-ba89-5eb91577b3f5 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:37.946322 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d77b829-e1bd-4624-ba89-5eb91577b3f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:37.946610 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d77b829-e1bd-4624-ba89-5eb91577b3f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:37.953154 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9b45e9-a671-4449-9f53-95b98331b937 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:37.958423 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d77b829-e1bd-4624-ba89-5eb91577b3f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:37.959877 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d77b829-e1bd-4624-ba89-5eb91577b3f5 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:37.973905 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-7d77b829-e1bd-4624-ba89-5eb91577b3f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:20:37.973905 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:37.973905 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:37.973905 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:37.973905 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:37.973905 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:37.973905 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:37.973905 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:37.973905 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:37.973905 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:37.973905 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:37.973905 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:37.973905 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:20:37.973905 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:20:37.973905 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:20:37.974769 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:20:37.974769 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:37.974769 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:37.974769 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:37.974769 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:37.974769 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:37.974769 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:37.974769 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:37.974769 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:37.974769 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:37.974769 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:37.974769 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:37.974769 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:37.974769 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:37.974769 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:37.974769 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:37.974769 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:37.975639 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:20:37.975639 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:20:37.975639 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:20:37.975639 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:20:37.975639 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 249/495] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:20:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:20:37.976579 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd31cac4-11c8-480f-b4fc-4bb7f30264fa None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:37.976802 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd31cac4-11c8-480f-b4fc-4bb7f30264fa None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:37.977102 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd31cac4-11c8-480f-b4fc-4bb7f30264fa None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:37.978333 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 250/496] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:20:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:20:37.979025 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd31cac4-11c8-480f-b4fc-4bb7f30264fa None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:37.983724 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:37.992885 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbeb0e3-1dfc-46af-ba1e-700241a87f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:37.993471 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbeb0e3-1dfc-46af-ba1e-700241a87f4e None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:37.993765 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbeb0e3-1dfc-46af-ba1e-700241a87f4e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:37.994047 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbeb0e3-1dfc-46af-ba1e-700241a87f4e None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:37.994849 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbeb0e3-1dfc-46af-ba1e-700241a87f4e None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:37.995168 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbeb0e3-1dfc-46af-ba1e-700241a87f4e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:37.997185 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbeb0e3-1dfc-46af-ba1e-700241a87f4e None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:38.033906 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 247/497] 10.222.0.26 () {56 vars in 1137 bytes} [Sat Aug 6 01:20:37 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:38.035300 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84556a25-e3c1-4b85-ba83-36f8343b1bbc None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:38.035585 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84556a25-e3c1-4b85-ba83-36f8343b1bbc None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:38.035874 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84556a25-e3c1-4b85-ba83-36f8343b1bbc None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:38.037122 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 248/498] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:20:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:20:38.038621 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84556a25-e3c1-4b85-ba83-36f8343b1bbc None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:38.088243 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd31cac4-11c8-480f-b4fc-4bb7f30264fa None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:38.122288 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46eb52c-b4b8-40cb-8585-304a1978a212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:38.122909 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46eb52c-b4b8-40cb-8585-304a1978a212 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:38.123192 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46eb52c-b4b8-40cb-8585-304a1978a212 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:38.123470 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46eb52c-b4b8-40cb-8585-304a1978a212 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:38.124191 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46eb52c-b4b8-40cb-8585-304a1978a212 None admin] RBAC: Adding query filter params (name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:38.124498 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46eb52c-b4b8-40cb-8585-304a1978a212 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:38.125902 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46eb52c-b4b8-40cb-8585-304a1978a212 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:38.130975 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:38.134021 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 251/499] 10.222.0.26 () {56 vars in 1103 bytes} [Sat Aug 6 01:20:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:38.135184 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f938622e-e635-4138-983a-4cbe9f0445bf None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:38.135481 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f938622e-e635-4138-983a-4cbe9f0445bf None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:38.135758 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f938622e-e635-4138-983a-4cbe9f0445bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:38.143190 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84556a25-e3c1-4b85-ba83-36f8343b1bbc None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:38.154178 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-f938622e-e635-4138-983a-4cbe9f0445bf None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:20:38.155372 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-f938622e-e635-4138-983a-4cbe9f0445bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:20:38.177509 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d3d34d-840c-4907-9189-db358026dead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:38.178129 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d3d34d-840c-4907-9189-db358026dead None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:38.178410 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d3d34d-840c-4907-9189-db358026dead None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:38.178687 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d3d34d-840c-4907-9189-db358026dead None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:38.179372 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d3d34d-840c-4907-9189-db358026dead None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:38.179675 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d3d34d-840c-4907-9189-db358026dead None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:38.181039 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d3d34d-840c-4907-9189-db358026dead None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:38.192032 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 249/500] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Aug 6 01:20:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:38.192814 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d3d34d-840c-4907-9189-db358026dead None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:38.215183 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:38.222710 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 252/501] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:20:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:20:38.223922 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eab3ace-540d-40f4-8cc0-07c242ebdb3d None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:38.224204 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eab3ace-540d-40f4-8cc0-07c242ebdb3d None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:38.224468 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eab3ace-540d-40f4-8cc0-07c242ebdb3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:38.243144 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-6eab3ace-540d-40f4-8cc0-07c242ebdb3d None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:20:38.244251 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-6eab3ace-540d-40f4-8cc0-07c242ebdb3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:20:38.298671 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d3d34d-840c-4907-9189-db358026dead None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:38.311218 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 253/502] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:20:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:20:38.312046 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eab3ace-540d-40f4-8cc0-07c242ebdb3d None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:38.337242 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd3f894-c862-42c4-8b98-524a84691ab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:38.337604 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd3f894-c862-42c4-8b98-524a84691ab9 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:38.337891 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd3f894-c862-42c4-8b98-524a84691ab9 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:38.338173 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd3f894-c862-42c4-8b98-524a84691ab9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:38.357245 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:38.407613 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd3f894-c862-42c4-8b98-524a84691ab9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:38.409144 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd3f894-c862-42c4-8b98-524a84691ab9 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:38.413175 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eab3ace-540d-40f4-8cc0-07c242ebdb3d None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:38.440016 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:38.449989 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5242b4f9-c70d-4768-804f-e4b53dc57838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:38.450634 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5242b4f9-c70d-4768-804f-e4b53dc57838 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:38.450918 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5242b4f9-c70d-4768-804f-e4b53dc57838 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:38.451204 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5242b4f9-c70d-4768-804f-e4b53dc57838 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:38.461510 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5242b4f9-c70d-4768-804f-e4b53dc57838 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:38.462987 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5242b4f9-c70d-4768-804f-e4b53dc57838 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:38.478009 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-5242b4f9-c70d-4768-804f-e4b53dc57838 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:20:38.478009 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:38.478009 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:38.478009 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:38.478009 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:38.478009 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:38.478009 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:38.478009 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:38.478009 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:38.478009 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:38.478009 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:38.478009 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:38.478009 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:20:38.478009 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:20:38.478009 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:20:38.479041 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:20:38.479041 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:38.479041 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:38.479041 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:38.479041 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:38.479041 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:38.479041 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:38.479041 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:38.479041 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:38.479041 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:38.479041 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:38.479041 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:38.479041 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:38.479041 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:38.479041 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:38.479041 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:38.479041 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:38.480255 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:20:38.480255 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:20:38.480255 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:20:38.480255 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:20:38.480255 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 254/503] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:20:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:20:38.480255 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5242b4f9-c70d-4768-804f-e4b53dc57838 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:38.514967 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 250/504] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Aug 6 01:20:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 01:20:38.515730 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd3f894-c862-42c4-8b98-524a84691ab9 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:38.563967 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:38.585003 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5242b4f9-c70d-4768-804f-e4b53dc57838 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:38.612995 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd3f894-c862-42c4-8b98-524a84691ab9 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:38.617453 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24ed0fd-7c5f-4bfb-98ac-b0a851c6670b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:38.618088 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24ed0fd-7c5f-4bfb-98ac-b0a851c6670b None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:38.618368 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24ed0fd-7c5f-4bfb-98ac-b0a851c6670b None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:38.618642 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24ed0fd-7c5f-4bfb-98ac-b0a851c6670b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:38.627245 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24ed0fd-7c5f-4bfb-98ac-b0a851c6670b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:38.628673 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24ed0fd-7c5f-4bfb-98ac-b0a851c6670b None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:38.640140 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-b24ed0fd-7c5f-4bfb-98ac-b0a851c6670b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 06 01:20:38.640140 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:38.640140 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:38.640140 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:38.640140 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:38.640140 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:38.640140 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:38.640140 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:38.640140 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:38.640140 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:38.640140 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:38.640140 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:38.640140 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 01:20:38.640140 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 01:20:38.640140 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 01:20:38.640140 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 01:20:38.641026 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:38.641026 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:38.641026 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 01:20:38.641026 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:20:38.641026 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 01:20:38.641026 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:20:38.641026 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:38.641026 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:38.641026 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:38.641026 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:38.641026 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:38.641026 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:38.641026 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:38.641026 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:38.641026 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:38.641026 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:38.641026 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:38.641026 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:38.642412 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:38.642412 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:38.642412 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 01:20:38.642412 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 01:20:38.642412 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 01:20:38.642412 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 01:20:38.642412 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 01:20:38.642412 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 01:20:38.642412 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 06 01:20:38.642412 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:20:38.642412 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 255/505] 10.222.0.26 () {56 vars in 1093 bytes} [Sat Aug 6 01:20:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:20:38.642412 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24ed0fd-7c5f-4bfb-98ac-b0a851c6670b None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:38.650642 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c665a8-bf0c-49d7-bc84-0ad5a83601af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:38.651286 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c665a8-bf0c-49d7-bc84-0ad5a83601af None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:38.651571 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c665a8-bf0c-49d7-bc84-0ad5a83601af None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:38.651848 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c665a8-bf0c-49d7-bc84-0ad5a83601af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:38.661918 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c665a8-bf0c-49d7-bc84-0ad5a83601af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:38.663398 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c665a8-bf0c-49d7-bc84-0ad5a83601af None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:38.677733 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-61c665a8-bf0c-49d7-bc84-0ad5a83601af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:20:38.677733 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:38.677733 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:38.677733 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:38.677733 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:38.677733 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:38.677733 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:38.677733 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:38.677733 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:38.677733 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:38.677733 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:38.677733 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:38.677733 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:20:38.677733 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:20:38.677733 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:20:38.678607 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:20:38.678607 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:38.678607 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:38.678607 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:38.678607 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:38.678607 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:38.678607 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:38.678607 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:38.678607 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:38.678607 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:38.678607 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:38.678607 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:38.678607 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:38.678607 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:38.678607 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:38.678607 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:38.678607 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:38.679539 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:20:38.679539 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:20:38.679539 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:20:38.679539 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:20:38.679539 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 251/506] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:20:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:20:38.679798 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c665a8-bf0c-49d7-bc84-0ad5a83601af None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:38.690331 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:38.751983 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24ed0fd-7c5f-4bfb-98ac-b0a851c6670b None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:38.781587 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c665a8-bf0c-49d7-bc84-0ad5a83601af None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:38.783032 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:38.785999 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddce63bb-6457-4d63-9d28-2044a410b038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:38.786638 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddce63bb-6457-4d63-9d28-2044a410b038 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:38.786951 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddce63bb-6457-4d63-9d28-2044a410b038 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:38.787231 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddce63bb-6457-4d63-9d28-2044a410b038 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:38.798742 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddce63bb-6457-4d63-9d28-2044a410b038 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:20:38.804048 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-ddce63bb-6457-4d63-9d28-2044a410b038 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 06 01:20:38.804048 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:38.804048 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:38.804048 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:38.804048 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:38.804048 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:38.804048 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:38.804048 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:38.804048 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:38.804048 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:38.804048 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:38.804048 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:38.804048 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:20:38.804048 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:20:38.804048 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:20:38.804048 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:20:38.805013 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:20:38.805013 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:20:38.805013 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:20:38.805013 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:20:38.805013 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:38.805013 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:38.805013 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:38.805013 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:38.805013 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:38.805013 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:38.805013 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:38.805013 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:38.805013 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:38.805013 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:38.805013 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:38.805013 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:38.805862 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:38.805862 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:38.805862 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:38.805862 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:38.805862 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:20:38.805862 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:20:38.805862 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:20:38.805862 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:20:38.805862 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:20:38.805862 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:20:38.805862 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 06 01:20:38.805862 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:20:38.805862 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 256/507] 10.222.0.26 () {56 vars in 1102 bytes} [Sat Aug 6 01:20:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:20:38.806513 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddce63bb-6457-4d63-9d28-2044a410b038 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:38.818406 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd243b8-25ce-449a-b922-4a5ea2489229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:38.819020 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd243b8-25ce-449a-b922-4a5ea2489229 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:38.819301 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd243b8-25ce-449a-b922-4a5ea2489229 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:38.819577 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd243b8-25ce-449a-b922-4a5ea2489229 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:38.820265 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd243b8-25ce-449a-b922-4a5ea2489229 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:38.820569 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd243b8-25ce-449a-b922-4a5ea2489229 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:38.821970 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd243b8-25ce-449a-b922-4a5ea2489229 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:38.834180 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 252/508] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Aug 6 01:20:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:38.834986 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd243b8-25ce-449a-b922-4a5ea2489229 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:38.907016 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:38.913189 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddce63bb-6457-4d63-9d28-2044a410b038 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:38.938277 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd243b8-25ce-449a-b922-4a5ea2489229 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:38.949631 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b43539-e2d0-4eef-83a2-2b88082e3d9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:38.950274 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b43539-e2d0-4eef-83a2-2b88082e3d9a None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:38.950560 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b43539-e2d0-4eef-83a2-2b88082e3d9a None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:38.950840 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b43539-e2d0-4eef-83a2-2b88082e3d9a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:38.961260 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b43539-e2d0-4eef-83a2-2b88082e3d9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:38.962706 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b43539-e2d0-4eef-83a2-2b88082e3d9a None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:38.970967 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ae823c-b621-4e43-bf73-5794bfa5013c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:38.971576 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ae823c-b621-4e43-bf73-5794bfa5013c None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:38.971853 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ae823c-b621-4e43-bf73-5794bfa5013c None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:38.972128 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ae823c-b621-4e43-bf73-5794bfa5013c None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:38.972904 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ae823c-b621-4e43-bf73-5794bfa5013c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:38.973248 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ae823c-b621-4e43-bf73-5794bfa5013c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:38.974648 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ae823c-b621-4e43-bf73-5794bfa5013c None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:38.977191 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-f1b43539-e2d0-4eef-83a2-2b88082e3d9a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:20:38.977191 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:38.977191 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:38.977191 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:38.977191 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:38.977191 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:38.977191 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:38.977191 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:38.977191 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:38.977191 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:38.977191 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:38.977191 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:38.977191 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:20:38.977191 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:20:38.977191 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:20:38.978238 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:20:38.978238 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:38.978238 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:38.978238 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:38.978238 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:38.978238 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:38.978238 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:38.978238 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:38.978238 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:38.978238 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:38.978238 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:38.978238 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:38.978238 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:38.978238 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:38.978238 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:38.978238 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:38.978238 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:38.979345 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:20:38.979345 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:20:38.979345 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:20:38.979345 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:20:38.979345 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 257/509] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:20:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:20:38.979345 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b43539-e2d0-4eef-83a2-2b88082e3d9a None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:39.001378 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 253/510] 10.222.0.26 () {56 vars in 1133 bytes} [Sat Aug 6 01:20:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:39.001378 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ae823c-b621-4e43-bf73-5794bfa5013c None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:39.020513 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:39.081649 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b43539-e2d0-4eef-83a2-2b88082e3d9a None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:39.098931 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ae823c-b621-4e43-bf73-5794bfa5013c None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:39.115779 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454a3e35-53b6-420a-aca3-fd37a3039da9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:39.116398 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454a3e35-53b6-420a-aca3-fd37a3039da9 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:39.116679 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454a3e35-53b6-420a-aca3-fd37a3039da9 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:39.116958 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454a3e35-53b6-420a-aca3-fd37a3039da9 None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:39.117695 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454a3e35-53b6-420a-aca3-fd37a3039da9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:39.118005 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454a3e35-53b6-420a-aca3-fd37a3039da9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:39.119378 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454a3e35-53b6-420a-aca3-fd37a3039da9 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:39.131232 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 258/511] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Aug 6 01:20:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:39.131941 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454a3e35-53b6-420a-aca3-fd37a3039da9 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:39.134463 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b09a8e3-a39a-4d2c-8bbc-dd73273be3bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:39.135076 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b09a8e3-a39a-4d2c-8bbc-dd73273be3bb None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:39.135393 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b09a8e3-a39a-4d2c-8bbc-dd73273be3bb None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:39.135635 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b09a8e3-a39a-4d2c-8bbc-dd73273be3bb None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:39.136344 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b09a8e3-a39a-4d2c-8bbc-dd73273be3bb None admin] RBAC: Adding query filter params (name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:39.137385 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b09a8e3-a39a-4d2c-8bbc-dd73273be3bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:39.138751 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b09a8e3-a39a-4d2c-8bbc-dd73273be3bb None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:39.145435 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:39.146885 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 254/512] 10.222.0.26 () {56 vars in 1103 bytes} [Sat Aug 6 01:20:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:39.147699 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b09a8e3-a39a-4d2c-8bbc-dd73273be3bb None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:39.229026 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:39.237192 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454a3e35-53b6-420a-aca3-fd37a3039da9 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:39.248531 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b09a8e3-a39a-4d2c-8bbc-dd73273be3bb None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:39.271401 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9361341-0e31-4d9d-8041-ed3dbf3654d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:39.272041 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9361341-0e31-4d9d-8041-ed3dbf3654d4 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:39.272324 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9361341-0e31-4d9d-8041-ed3dbf3654d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:39.272603 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9361341-0e31-4d9d-8041-ed3dbf3654d4 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:39.280700 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9361341-0e31-4d9d-8041-ed3dbf3654d4 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:39.281490 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd6326f-1f01-4350-a16f-96e4b883a246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:39.282104 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd6326f-1f01-4350-a16f-96e4b883a246 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:39.282196 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9361341-0e31-4d9d-8041-ed3dbf3654d4 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:39.282386 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd6326f-1f01-4350-a16f-96e4b883a246 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:39.282662 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd6326f-1f01-4350-a16f-96e4b883a246 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:39.283348 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd6326f-1f01-4350-a16f-96e4b883a246 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:39.283653 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd6326f-1f01-4350-a16f-96e4b883a246 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:39.284994 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd6326f-1f01-4350-a16f-96e4b883a246 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:39.294716 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-b9361341-0e31-4d9d-8041-ed3dbf3654d4 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 06 01:20:39.294716 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:39.294716 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:39.294716 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:39.294716 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:39.294716 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:39.294716 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:39.294716 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:39.294716 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:39.294716 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:39.294716 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:39.294716 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:39.294716 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 01:20:39.294716 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 01:20:39.294716 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 01:20:39.294716 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 01:20:39.295609 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:39.295609 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:39.295609 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 01:20:39.295609 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:20:39.295609 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 01:20:39.295609 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:20:39.295609 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:39.295609 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:39.295609 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:39.295609 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:39.295609 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:39.295609 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:39.295609 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:39.295609 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:39.295609 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:39.295609 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:39.295609 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:39.295609 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:39.296489 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:39.296489 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:39.296489 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 01:20:39.296489 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 01:20:39.296489 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 01:20:39.296489 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 01:20:39.296489 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 01:20:39.296489 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 01:20:39.296489 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 06 01:20:39.296489 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:20:39.296489 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 259/513] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Aug 6 01:20:39 2022] GET /identity/v3/users/cinder => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:20:39.297107 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9361341-0e31-4d9d-8041-ed3dbf3654d4 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:39.300773 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 255/514] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Aug 6 01:20:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:39.301574 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd6326f-1f01-4350-a16f-96e4b883a246 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:39.372294 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:39.402512 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9361341-0e31-4d9d-8041-ed3dbf3654d4 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:39.403626 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd6326f-1f01-4350-a16f-96e4b883a246 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:39.437301 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffc0e8f-9c49-455b-aea7-c63c14187697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:39.437919 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffc0e8f-9c49-455b-aea7-c63c14187697 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:39.438201 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffc0e8f-9c49-455b-aea7-c63c14187697 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:39.438475 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffc0e8f-9c49-455b-aea7-c63c14187697 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:39.438686 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63019c1a-64a6-47c3-ba64-457e0ed304f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:39.439312 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63019c1a-64a6-47c3-ba64-457e0ed304f3 None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:39.439597 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63019c1a-64a6-47c3-ba64-457e0ed304f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:39.439877 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63019c1a-64a6-47c3-ba64-457e0ed304f3 None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:39.440560 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63019c1a-64a6-47c3-ba64-457e0ed304f3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:39.442025 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63019c1a-64a6-47c3-ba64-457e0ed304f3 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:39.458070 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:39.477079 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 256/515] 10.222.0.26 () {60 vars in 1132 bytes} [Sat Aug 6 01:20:39 2022] POST /identity/v3/users => generated 247 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:20:39.477079 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63019c1a-64a6-47c3-ba64-457e0ed304f3 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:39.513135 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffc0e8f-9c49-455b-aea7-c63c14187697 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:39.514653 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffc0e8f-9c49-455b-aea7-c63c14187697 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:39.577434 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63019c1a-64a6-47c3-ba64-457e0ed304f3 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:39.589220 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:39.617039 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1720751-90fd-4d9a-b09d-605ce5fac198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:39.617727 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1720751-90fd-4d9a-b09d-605ce5fac198 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:39.617981 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1720751-90fd-4d9a-b09d-605ce5fac198 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:39.618255 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1720751-90fd-4d9a-b09d-605ce5fac198 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:39.628069 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 260/516] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Aug 6 01:20:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 01:20:39.628688 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffc0e8f-9c49-455b-aea7-c63c14187697 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:39.629389 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1720751-90fd-4d9a-b09d-605ce5fac198 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:39.630840 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1720751-90fd-4d9a-b09d-605ce5fac198 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:39.648047 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-e1720751-90fd-4d9a-b09d-605ce5fac198 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:20:39.648047 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:39.648047 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:39.648047 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:39.648047 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:39.648047 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:39.648047 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:39.648047 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:39.648047 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:39.648047 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:39.648047 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:39.648047 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:39.648047 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:20:39.648047 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:20:39.648047 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:20:39.648960 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:20:39.648960 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:39.648960 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:39.648960 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:39.648960 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:39.648960 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:39.648960 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:39.648960 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:39.648960 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:39.648960 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:39.648960 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:39.648960 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:39.648960 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:39.648960 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:39.648960 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:39.648960 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:39.648960 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:39.651177 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:20:39.651177 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:20:39.651177 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:20:39.651177 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:20:39.651177 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 257/517] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:20:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:20:39.651177 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1720751-90fd-4d9a-b09d-605ce5fac198 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:39.705818 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:39.741350 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffc0e8f-9c49-455b-aea7-c63c14187697 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:39.765126 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1720751-90fd-4d9a-b09d-605ce5fac198 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:39.777744 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746850e2-0742-4cbe-8e15-b0c33d806721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:39.778470 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746850e2-0742-4cbe-8e15-b0c33d806721 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:39.778758 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746850e2-0742-4cbe-8e15-b0c33d806721 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:39.779030 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746850e2-0742-4cbe-8e15-b0c33d806721 None admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:39.779849 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746850e2-0742-4cbe-8e15-b0c33d806721 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:39.780156 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746850e2-0742-4cbe-8e15-b0c33d806721 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:39.781616 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746850e2-0742-4cbe-8e15-b0c33d806721 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:39.803071 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d138daef-d708-4882-a579-becdc37b6414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:39.803960 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d138daef-d708-4882-a579-becdc37b6414 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:39.804405 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d138daef-d708-4882-a579-becdc37b6414 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:39.804830 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d138daef-d708-4882-a579-becdc37b6414 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:39.812286 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d138daef-d708-4882-a579-becdc37b6414 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:39.813983 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d138daef-d708-4882-a579-becdc37b6414 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:39.817973 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 261/518] 10.222.0.26 () {56 vars in 1137 bytes} [Sat Aug 6 01:20:39 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:39.818821 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746850e2-0742-4cbe-8e15-b0c33d806721 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:39.829967 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-d138daef-d708-4882-a579-becdc37b6414 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:20:39.829967 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:39.829967 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:39.829967 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:39.829967 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:39.829967 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:39.829967 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:39.829967 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:39.829967 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:39.829967 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:39.829967 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:39.829967 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:39.829967 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 01:20:39.829967 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 01:20:39.829967 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 01:20:39.830950 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 01:20:39.830950 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:39.830950 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:39.830950 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:39.830950 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:39.830950 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:39.830950 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:39.830950 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:39.830950 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:39.830950 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:39.830950 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:39.830950 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:39.830950 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:39.830950 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:39.830950 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:39.830950 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:39.830950 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:39.831847 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 01:20:39.831847 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 01:20:39.831847 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 01:20:39.831847 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 01:20:39.831847 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 01:20:39.831847 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 01:20:39.831847 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:20:39.831847 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:20:39.831847 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 258/519] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Aug 6 01:20:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:20:39.832343 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d138daef-d708-4882-a579-becdc37b6414 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:39.855881 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:39.936441 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746850e2-0742-4cbe-8e15-b0c33d806721 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:39.941182 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d138daef-d708-4882-a579-becdc37b6414 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:39.952366 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:39.975791 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9140b2d3-3839-4698-96f9-bcdfc66b2386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:39.976694 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9140b2d3-3839-4698-96f9-bcdfc66b2386 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:39.976977 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9140b2d3-3839-4698-96f9-bcdfc66b2386 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:39.977288 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9140b2d3-3839-4698-96f9-bcdfc66b2386 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:39.982183 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af972a1-e181-4644-9009-182963e29efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:39.982810 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af972a1-e181-4644-9009-182963e29efd None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:39.983098 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af972a1-e181-4644-9009-182963e29efd None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:39.983378 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af972a1-e181-4644-9009-182963e29efd None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:39.984071 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af972a1-e181-4644-9009-182963e29efd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:39.984808 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af972a1-e181-4644-9009-182963e29efd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:39.986678 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af972a1-e181-4644-9009-182963e29efd None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:39.988030 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9140b2d3-3839-4698-96f9-bcdfc66b2386 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:39.989512 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9140b2d3-3839-4698-96f9-bcdfc66b2386 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:40.003853 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 259/520] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Aug 6 01:20:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:40.003853 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af972a1-e181-4644-9009-182963e29efd None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:40.011302 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-9140b2d3-3839-4698-96f9-bcdfc66b2386 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:20:40.011302 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:40.011302 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:40.011302 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:40.011302 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:40.011302 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:40.011302 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:40.011302 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:40.011302 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:40.011302 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:40.011302 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:40.011302 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:40.011302 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:20:40.011302 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:20:40.011302 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:20:40.012192 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:20:40.012192 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:40.012192 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:40.012192 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:40.012192 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:40.012192 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:40.012192 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:40.012192 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:40.012192 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:40.012192 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:40.012192 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:40.012192 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:40.012192 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:40.012192 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:40.012192 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:40.012192 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:40.012192 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:40.013039 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:20:40.013039 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:20:40.013039 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:20:40.013039 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:20:40.017078 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 262/521] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:20:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:20:40.017931 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9140b2d3-3839-4698-96f9-bcdfc66b2386 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:40.101967 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:40.116712 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af972a1-e181-4644-9009-182963e29efd None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:40.126401 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9140b2d3-3839-4698-96f9-bcdfc66b2386 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:40.153499 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00a8844-8978-4318-94a3-b6fdc48cb6fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:40.154124 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00a8844-8978-4318-94a3-b6fdc48cb6fe None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:40.154409 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00a8844-8978-4318-94a3-b6fdc48cb6fe None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:40.154681 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00a8844-8978-4318-94a3-b6fdc48cb6fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:40.163377 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1aa559-de19-4062-8b16-a4370b161d6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:40.163999 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1aa559-de19-4062-8b16-a4370b161d6f None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:40.164281 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1aa559-de19-4062-8b16-a4370b161d6f None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:40.164556 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1aa559-de19-4062-8b16-a4370b161d6f None admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:40.165333 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1aa559-de19-4062-8b16-a4370b161d6f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:40.165642 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1aa559-de19-4062-8b16-a4370b161d6f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:40.167066 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1aa559-de19-4062-8b16-a4370b161d6f None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:40.190814 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 263/522] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Aug 6 01:20:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:40.191663 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1aa559-de19-4062-8b16-a4370b161d6f None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:40.194963 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:40.226626 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00a8844-8978-4318-94a3-b6fdc48cb6fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:40.229096 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00a8844-8978-4318-94a3-b6fdc48cb6fe None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:40.296473 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1aa559-de19-4062-8b16-a4370b161d6f None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:40.331899 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91600615-7e93-4cc1-a2b4-85ae37dfb76d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:40.332535 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91600615-7e93-4cc1-a2b4-85ae37dfb76d None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:40.332814 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91600615-7e93-4cc1-a2b4-85ae37dfb76d None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:40.333112 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91600615-7e93-4cc1-a2b4-85ae37dfb76d None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:40.341011 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91600615-7e93-4cc1-a2b4-85ae37dfb76d None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:40.342475 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91600615-7e93-4cc1-a2b4-85ae37dfb76d None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:40.345534 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:40.348642 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 260/523] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Aug 6 01:20:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 01:20:40.349572 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00a8844-8978-4318-94a3-b6fdc48cb6fe None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:40.356055 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-91600615-7e93-4cc1-a2b4-85ae37dfb76d None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 06 01:20:40.356055 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:40.356055 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:40.356055 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:40.356055 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:40.356055 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:40.356055 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:40.356055 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:40.356055 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:40.356055 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:40.356055 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:40.356055 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:40.356055 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 01:20:40.356055 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 01:20:40.356055 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 01:20:40.356055 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 01:20:40.357267 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:40.357267 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:40.357267 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 01:20:40.357267 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:20:40.357267 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 01:20:40.357267 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:20:40.357267 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:40.357267 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:40.357267 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:40.357267 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:40.357267 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:40.357267 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:40.357267 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:40.357267 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:40.357267 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:40.357267 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:40.357267 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:40.357267 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:40.358481 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:40.358481 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:40.358481 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 01:20:40.358481 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 01:20:40.358481 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 01:20:40.358481 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 01:20:40.358481 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 01:20:40.358481 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 01:20:40.358481 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 06 01:20:40.358481 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:20:40.361114 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 264/524] 10.222.0.26 () {56 vars in 1102 bytes} [Sat Aug 6 01:20:40 2022] GET /identity/v3/users/neutron => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:20:40.361835 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91600615-7e93-4cc1-a2b4-85ae37dfb76d None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:40.432593 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:40.466697 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00a8844-8978-4318-94a3-b6fdc48cb6fe None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:40.475243 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91600615-7e93-4cc1-a2b4-85ae37dfb76d None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:40.503210 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f6903b-492d-477f-a7be-bc2816f7c74b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:40.503846 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f6903b-492d-477f-a7be-bc2816f7c74b None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:40.504132 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f6903b-492d-477f-a7be-bc2816f7c74b None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:40.504416 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f6903b-492d-477f-a7be-bc2816f7c74b None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:40.505171 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f6903b-492d-477f-a7be-bc2816f7c74b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:40.505482 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f6903b-492d-477f-a7be-bc2816f7c74b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:40.506875 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f6903b-492d-477f-a7be-bc2816f7c74b None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:40.511612 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229fcf52-7c9f-42a6-88e9-e4957b325fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:40.512514 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229fcf52-7c9f-42a6-88e9-e4957b325fb8 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:40.512794 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229fcf52-7c9f-42a6-88e9-e4957b325fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:40.513085 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229fcf52-7c9f-42a6-88e9-e4957b325fb8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:40.514072 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229fcf52-7c9f-42a6-88e9-e4957b325fb8 None admin] RBAC: Adding query filter params (role.id=c9ce798acd4b482f8f391c6af97eb7a4, scope.project.id=d9d68d310bf24449a5fe20d2dc264a58, user.id=abb76a5c8ef74bf2b18c56074daaad58) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:40.514178 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229fcf52-7c9f-42a6-88e9-e4957b325fb8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:40.515593 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229fcf52-7c9f-42a6-88e9-e4957b325fb8 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:40.521168 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 265/525] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Aug 6 01:20:40 2022] GET /identity/v3/role_assignments?user.id=abb76a5c8ef74bf2b18c56074daaad58&scope.project.id=d9d68d310bf24449a5fe20d2dc264a58&role.id=c9ce798acd4b482f8f391c6af97eb7a4 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:40.522201 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229fcf52-7c9f-42a6-88e9-e4957b325fb8 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:40.524661 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 261/526] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Aug 6 01:20:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:40.525654 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f6903b-492d-477f-a7be-bc2816f7c74b None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:40.609660 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:40.648766 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229fcf52-7c9f-42a6-88e9-e4957b325fb8 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:40.668879 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f6903b-492d-477f-a7be-bc2816f7c74b None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:40.687767 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e79d9b-9755-48f2-9a03-abec550b4cdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:40.688434 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e79d9b-9755-48f2-9a03-abec550b4cdf None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:40.688717 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e79d9b-9755-48f2-9a03-abec550b4cdf None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:40.688996 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e79d9b-9755-48f2-9a03-abec550b4cdf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:40.701762 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e79d9b-9755-48f2-9a03-abec550b4cdf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:40.703254 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e79d9b-9755-48f2-9a03-abec550b4cdf None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:40.711179 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:40.719745 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94350de-f9b3-4783-a754-b9dc6b6b55e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:40.720737 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94350de-f9b3-4783-a754-b9dc6b6b55e3 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:40.721240 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94350de-f9b3-4783-a754-b9dc6b6b55e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:40.721782 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94350de-f9b3-4783-a754-b9dc6b6b55e3 None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:40.721898 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-e3e79d9b-9755-48f2-9a03-abec550b4cdf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:20:40.721898 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:40.721898 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:40.721898 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:40.721898 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:40.721898 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:40.721898 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:40.721898 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:40.721898 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:40.721898 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:40.721898 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:40.721898 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:40.721898 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:20:40.721898 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:20:40.721898 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:20:40.722694 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:20:40.722694 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:40.722694 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:40.722694 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:40.722694 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:40.722694 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:40.722694 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:40.722694 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:40.722694 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:40.722694 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:40.722694 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:40.722694 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:40.722694 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:40.722694 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:40.722694 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:40.722694 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:40.722694 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:40.723531 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:20:40.723531 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:20:40.723531 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:20:40.723531 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:20:40.723531 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 266/527] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:20:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:20:40.724056 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e79d9b-9755-48f2-9a03-abec550b4cdf None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:40.725004 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94350de-f9b3-4783-a754-b9dc6b6b55e3 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:40.725557 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94350de-f9b3-4783-a754-b9dc6b6b55e3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:40.727275 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94350de-f9b3-4783-a754-b9dc6b6b55e3 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:40.757185 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 262/528] 10.222.0.26 () {56 vars in 1139 bytes} [Sat Aug 6 01:20:40 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:40.759082 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94350de-f9b3-4783-a754-b9dc6b6b55e3 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:40.848309 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e79d9b-9755-48f2-9a03-abec550b4cdf None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:40.872138 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:40.899358 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94350de-f9b3-4783-a754-b9dc6b6b55e3 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:40.903636 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eacb6db-1f60-42c6-8a27-94cb957bfd9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:40.904286 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eacb6db-1f60-42c6-8a27-94cb957bfd9b None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:40.904567 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eacb6db-1f60-42c6-8a27-94cb957bfd9b None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:40.904840 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eacb6db-1f60-42c6-8a27-94cb957bfd9b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:40.912695 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eacb6db-1f60-42c6-8a27-94cb957bfd9b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:40.914363 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eacb6db-1f60-42c6-8a27-94cb957bfd9b None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:40.926511 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-4eacb6db-1f60-42c6-8a27-94cb957bfd9b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:20:40.926511 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:40.926511 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:40.926511 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:40.926511 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:40.926511 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:40.926511 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:40.926511 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:40.926511 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:40.926511 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:40.926511 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:40.926511 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:40.926511 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 01:20:40.926511 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 01:20:40.926511 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 01:20:40.927393 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 01:20:40.927393 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:40.927393 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:40.927393 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:40.927393 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:40.927393 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:40.927393 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:40.927393 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:40.927393 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:40.927393 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:40.927393 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:40.927393 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:40.927393 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:40.927393 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:40.927393 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:40.927393 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:40.927393 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:40.928235 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 01:20:40.928235 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 01:20:40.928235 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 01:20:40.928235 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 01:20:40.928235 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 01:20:40.928235 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 01:20:40.928235 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:20:40.928235 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:20:40.932607 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 267/529] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Aug 6 01:20:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:20:40.934229 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eacb6db-1f60-42c6-8a27-94cb957bfd9b None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:40.949944 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d7783e-f73f-475a-804f-0ea8fb4d7c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:40.950580 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d7783e-f73f-475a-804f-0ea8fb4d7c0b None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:40.950864 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d7783e-f73f-475a-804f-0ea8fb4d7c0b None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:40.951140 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d7783e-f73f-475a-804f-0ea8fb4d7c0b None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:40.951838 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d7783e-f73f-475a-804f-0ea8fb4d7c0b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:40.952141 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d7783e-f73f-475a-804f-0ea8fb4d7c0b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:40.953575 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d7783e-f73f-475a-804f-0ea8fb4d7c0b None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:40.971094 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 263/530] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Aug 6 01:20:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:40.972012 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d7783e-f73f-475a-804f-0ea8fb4d7c0b None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:41.024765 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:41.068301 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eacb6db-1f60-42c6-8a27-94cb957bfd9b None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:41.083043 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d7783e-f73f-475a-804f-0ea8fb4d7c0b None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:41.106379 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebdf99a-48c3-4ebb-a419-412afbba289d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:41.107363 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebdf99a-48c3-4ebb-a419-412afbba289d None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:41.108060 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebdf99a-48c3-4ebb-a419-412afbba289d None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:41.108540 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebdf99a-48c3-4ebb-a419-412afbba289d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:41.119460 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b078ccf-2ca8-496e-96cc-b3578e69a490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:41.120149 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b078ccf-2ca8-496e-96cc-b3578e69a490 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:41.120432 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b078ccf-2ca8-496e-96cc-b3578e69a490 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:41.120788 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b078ccf-2ca8-496e-96cc-b3578e69a490 None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:41.121647 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b078ccf-2ca8-496e-96cc-b3578e69a490 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:41.122258 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b078ccf-2ca8-496e-96cc-b3578e69a490 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:41.123392 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b078ccf-2ca8-496e-96cc-b3578e69a490 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:41.147188 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 264/531] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Aug 6 01:20:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:41.148105 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b078ccf-2ca8-496e-96cc-b3578e69a490 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:41.160703 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:41.186904 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebdf99a-48c3-4ebb-a419-412afbba289d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:41.188715 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebdf99a-48c3-4ebb-a419-412afbba289d None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:41.254081 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b078ccf-2ca8-496e-96cc-b3578e69a490 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:41.265408 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:41.289900 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f73bbe-90e3-475d-81a8-0d802837bd49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:41.290586 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f73bbe-90e3-475d-81a8-0d802837bd49 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:41.290870 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f73bbe-90e3-475d-81a8-0d802837bd49 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:41.291147 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f73bbe-90e3-475d-81a8-0d802837bd49 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:41.299276 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f73bbe-90e3-475d-81a8-0d802837bd49 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:41.304118 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f73bbe-90e3-475d-81a8-0d802837bd49 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:41.304648 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 268/532] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Aug 6 01:20:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 01:20:41.305809 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebdf99a-48c3-4ebb-a419-412afbba289d None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:41.317209 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-90f73bbe-90e3-475d-81a8-0d802837bd49 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:20:41.317209 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:41.317209 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:41.317209 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:41.317209 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:41.317209 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:41.317209 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:41.317209 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:41.317209 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:41.317209 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:41.317209 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:41.317209 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:41.317209 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 01:20:41.317209 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 01:20:41.317209 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 01:20:41.318098 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 01:20:41.318098 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:41.318098 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:41.318098 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:41.318098 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:41.318098 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:41.318098 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:41.318098 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:41.318098 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:41.318098 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:41.318098 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:41.318098 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:41.318098 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:41.318098 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:41.318098 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:41.318098 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:41.318098 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:41.318945 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 01:20:41.318945 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 01:20:41.318945 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 01:20:41.318945 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 01:20:41.318945 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 01:20:41.318945 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 01:20:41.318945 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:20:41.318945 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:20:41.323093 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 265/533] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Aug 6 01:20:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:20:41.323965 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f73bbe-90e3-475d-81a8-0d802837bd49 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:41.414226 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:41.424204 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebdf99a-48c3-4ebb-a419-412afbba289d None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:41.430548 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f73bbe-90e3-475d-81a8-0d802837bd49 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:41.461378 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf95d0e-32ae-4bb7-8c9b-513d71f162c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:41.461997 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf95d0e-32ae-4bb7-8c9b-513d71f162c5 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:41.462279 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf95d0e-32ae-4bb7-8c9b-513d71f162c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:41.463188 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf95d0e-32ae-4bb7-8c9b-513d71f162c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:41.464166 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf95d0e-32ae-4bb7-8c9b-513d71f162c5 None admin] RBAC: Adding query filter params (role.id=c9ce798acd4b482f8f391c6af97eb7a4, scope.project.id=d9d68d310bf24449a5fe20d2dc264a58, user.id=b87adc1d5bc240e99f95f6595b350be4) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:41.464635 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf95d0e-32ae-4bb7-8c9b-513d71f162c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:41.465779 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b913f34a-19ff-4c17-a9b8-dc1ddb0804d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:41.466421 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b913f34a-19ff-4c17-a9b8-dc1ddb0804d3 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:41.466704 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b913f34a-19ff-4c17-a9b8-dc1ddb0804d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:41.466856 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf95d0e-32ae-4bb7-8c9b-513d71f162c5 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:41.466980 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b913f34a-19ff-4c17-a9b8-dc1ddb0804d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:41.473268 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 269/534] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Aug 6 01:20:41 2022] GET /identity/v3/role_assignments?user.id=b87adc1d5bc240e99f95f6595b350be4&scope.project.id=d9d68d310bf24449a5fe20d2dc264a58&role.id=c9ce798acd4b482f8f391c6af97eb7a4 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:41.474097 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf95d0e-32ae-4bb7-8c9b-513d71f162c5 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:41.486610 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b913f34a-19ff-4c17-a9b8-dc1ddb0804d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:41.488142 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b913f34a-19ff-4c17-a9b8-dc1ddb0804d3 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:41.516284 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-b913f34a-19ff-4c17-a9b8-dc1ddb0804d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:20:41.516284 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:41.516284 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:41.516284 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:41.516284 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:41.516284 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:41.516284 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:41.516284 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:41.516284 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:41.516284 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:41.516284 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:41.516284 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:41.516284 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:20:41.516284 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:20:41.516284 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:20:41.517179 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:20:41.517179 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:41.517179 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:41.517179 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:41.517179 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:41.517179 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:41.517179 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:41.517179 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:41.517179 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:41.517179 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:41.517179 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:41.517179 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:41.517179 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:41.517179 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:41.517179 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:41.517179 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:41.517179 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:41.518032 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:20:41.518032 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:20:41.518032 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:20:41.518032 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:20:41.518032 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 266/535] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:20:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:20:41.524354 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b913f34a-19ff-4c17-a9b8-dc1ddb0804d3 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:41.555747 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:41.594098 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf95d0e-32ae-4bb7-8c9b-513d71f162c5 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:41.635546 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6517f91d-9cea-4bde-8f15-8f74aac1c0ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:41.636277 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6517f91d-9cea-4bde-8f15-8f74aac1c0ef None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:41.637224 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6517f91d-9cea-4bde-8f15-8f74aac1c0ef None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:41.637808 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6517f91d-9cea-4bde-8f15-8f74aac1c0ef None admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:41.638853 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6517f91d-9cea-4bde-8f15-8f74aac1c0ef None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:41.639162 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6517f91d-9cea-4bde-8f15-8f74aac1c0ef None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:41.640734 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6517f91d-9cea-4bde-8f15-8f74aac1c0ef None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:41.644582 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b913f34a-19ff-4c17-a9b8-dc1ddb0804d3 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:41.650603 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:41.656066 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 270/536] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Aug 6 01:20:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:41.657571 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30bfb61-01e9-4cfb-8ffd-3ab76abc6507 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:41.657870 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30bfb61-01e9-4cfb-8ffd-3ab76abc6507 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:41.658140 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30bfb61-01e9-4cfb-8ffd-3ab76abc6507 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:41.659638 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 271/537] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:20:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:20:41.660951 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30bfb61-01e9-4cfb-8ffd-3ab76abc6507 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:41.687309 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6e92a1-ee6b-4b09-9974-41a5fbb07c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:41.687994 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6e92a1-ee6b-4b09-9974-41a5fbb07c2e None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:41.688196 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6e92a1-ee6b-4b09-9974-41a5fbb07c2e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:41.688472 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6e92a1-ee6b-4b09-9974-41a5fbb07c2e None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:41.689201 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6e92a1-ee6b-4b09-9974-41a5fbb07c2e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:41.689505 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6e92a1-ee6b-4b09-9974-41a5fbb07c2e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:41.690879 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6e92a1-ee6b-4b09-9974-41a5fbb07c2e None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:41.705380 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 267/538] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Aug 6 01:20:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:41.706783 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddb9063-fbe7-4bf1-b1ba-40a186edc901 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:41.707069 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddb9063-fbe7-4bf1-b1ba-40a186edc901 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:41.707399 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddb9063-fbe7-4bf1-b1ba-40a186edc901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:41.729014 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-1ddb9063-fbe7-4bf1-b1ba-40a186edc901 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:20:41.731435 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-1ddb9063-fbe7-4bf1-b1ba-40a186edc901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:20:41.788373 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30bfb61-01e9-4cfb-8ffd-3ab76abc6507 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:41.804549 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 268/539] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:20:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:20:41.805652 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddb9063-fbe7-4bf1-b1ba-40a186edc901 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:41.828932 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:41.833000 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3685d88-f14a-4bd1-b5ae-5504a1de75ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:41.833854 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3685d88-f14a-4bd1-b5ae-5504a1de75ae None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:41.834275 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3685d88-f14a-4bd1-b5ae-5504a1de75ae None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:41.834687 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3685d88-f14a-4bd1-b5ae-5504a1de75ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:41.836038 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3685d88-f14a-4bd1-b5ae-5504a1de75ae None admin] RBAC: Adding query filter params (role.id=c9ce798acd4b482f8f391c6af97eb7a4, scope.project.id=d9d68d310bf24449a5fe20d2dc264a58, user.id=1be1e6173e374a60b9e1fbf51a1ab571) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:41.836788 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3685d88-f14a-4bd1-b5ae-5504a1de75ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:41.838995 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3685d88-f14a-4bd1-b5ae-5504a1de75ae None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:41.847669 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 272/540] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Aug 6 01:20:41 2022] GET /identity/v3/role_assignments?user.id=1be1e6173e374a60b9e1fbf51a1ab571&scope.project.id=d9d68d310bf24449a5fe20d2dc264a58&role.id=c9ce798acd4b482f8f391c6af97eb7a4 => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:20:41.860322 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3685d88-f14a-4bd1-b5ae-5504a1de75ae None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:41.937867 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddb9063-fbe7-4bf1-b1ba-40a186edc901 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:41.979296 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f96c86-2840-4c99-84cd-0616d4cda035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:41.979952 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f96c86-2840-4c99-84cd-0616d4cda035 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:41.980293 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f96c86-2840-4c99-84cd-0616d4cda035 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:41.980690 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f96c86-2840-4c99-84cd-0616d4cda035 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:42.002813 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f96c86-2840-4c99-84cd-0616d4cda035 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:42.004593 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f96c86-2840-4c99-84cd-0616d4cda035 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:42.022423 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-d4f96c86-2840-4c99-84cd-0616d4cda035 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:20:42.022423 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:42.022423 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:42.022423 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:42.022423 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:42.022423 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:42.022423 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:42.022423 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:42.022423 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:42.022423 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:42.022423 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:42.022423 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:42.022423 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 01:20:42.022423 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 01:20:42.022423 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 01:20:42.023291 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 01:20:42.023291 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:42.023291 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:42.023291 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:42.023291 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:42.023291 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:42.023291 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:42.023291 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:42.023291 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:42.023291 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:42.023291 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:42.023291 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:42.023291 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:42.023291 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:42.023291 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:42.023291 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:42.023291 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:42.024110 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 01:20:42.024110 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 01:20:42.024110 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 01:20:42.024110 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 01:20:42.024110 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 01:20:42.024110 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 01:20:42.024110 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:20:42.024110 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:20:42.026515 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 269/541] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Aug 6 01:20:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:20:42.031639 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:42.032961 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f96c86-2840-4c99-84cd-0616d4cda035 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:42.099894 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3685d88-f14a-4bd1-b5ae-5504a1de75ae None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:42.111998 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:42.150228 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09aecb3-d5d4-4822-8977-0829ed1e1f6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:42.150883 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09aecb3-d5d4-4822-8977-0829ed1e1f6d None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:42.151167 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09aecb3-d5d4-4822-8977-0829ed1e1f6d None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:42.151445 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09aecb3-d5d4-4822-8977-0829ed1e1f6d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:42.170044 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f96c86-2840-4c99-84cd-0616d4cda035 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:42.172910 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09aecb3-d5d4-4822-8977-0829ed1e1f6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:20:42.182630 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-d09aecb3-d5d4-4822-8977-0829ed1e1f6d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 06 01:20:42.182630 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:42.182630 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:42.182630 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:42.182630 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:42.182630 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:42.182630 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:42.182630 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:42.182630 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:42.182630 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:42.182630 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:42.182630 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:42.182630 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:20:42.182630 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:20:42.182630 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:20:42.182630 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:20:42.184199 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:20:42.184199 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:20:42.184199 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:20:42.184199 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:20:42.184199 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:42.184199 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:42.184199 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:42.184199 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:42.184199 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:42.184199 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:42.184199 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:42.184199 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:42.184199 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:42.184199 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:42.184199 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:42.184199 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:42.185013 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:42.185013 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:42.185013 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:42.185013 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:42.185013 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:20:42.185013 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:20:42.185013 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:20:42.185013 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:20:42.185013 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:20:42.185013 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:20:42.185013 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 06 01:20:42.185013 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:20:42.186628 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 273/542] 10.222.0.26 () {56 vars in 1102 bytes} [Sat Aug 6 01:20:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:20:42.191791 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09aecb3-d5d4-4822-8977-0829ed1e1f6d None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:42.235695 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3081220-bc61-4826-b045-1e8a63e98a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:42.236333 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3081220-bc61-4826-b045-1e8a63e98a6d None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:42.236619 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3081220-bc61-4826-b045-1e8a63e98a6d None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:42.236897 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3081220-bc61-4826-b045-1e8a63e98a6d None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:42.237785 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3081220-bc61-4826-b045-1e8a63e98a6d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:42.238365 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3081220-bc61-4826-b045-1e8a63e98a6d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:42.239770 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3081220-bc61-4826-b045-1e8a63e98a6d None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:42.263256 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 270/543] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Aug 6 01:20:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:42.269869 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3081220-bc61-4826-b045-1e8a63e98a6d None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:42.419003 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3081220-bc61-4826-b045-1e8a63e98a6d None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:42.419896 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09aecb3-d5d4-4822-8977-0829ed1e1f6d None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:42.454968 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:42.469203 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e632bd35-e5a8-434f-95be-adf07fadf7e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:42.469203 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e632bd35-e5a8-434f-95be-adf07fadf7e3 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:42.469203 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e632bd35-e5a8-434f-95be-adf07fadf7e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:42.469203 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e632bd35-e5a8-434f-95be-adf07fadf7e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:42.469626 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e632bd35-e5a8-434f-95be-adf07fadf7e3 None admin] RBAC: Adding query filter params (role.id=c9ce798acd4b482f8f391c6af97eb7a4, scope.project.id=d9d68d310bf24449a5fe20d2dc264a58, user.id=4ac0e31c84b4445fa167bbe4954066c2) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:42.469626 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e632bd35-e5a8-434f-95be-adf07fadf7e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:42.469626 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e632bd35-e5a8-434f-95be-adf07fadf7e3 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:42.471289 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:42.472196 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 271/544] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Aug 6 01:20:42 2022] GET /identity/v3/role_assignments?user.id=4ac0e31c84b4445fa167bbe4954066c2&scope.project.id=d9d68d310bf24449a5fe20d2dc264a58&role.id=c9ce798acd4b482f8f391c6af97eb7a4 => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:20:42.491788 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916d8895-bd92-4369-9060-c367302aacad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:42.492695 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916d8895-bd92-4369-9060-c367302aacad None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:42.493151 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916d8895-bd92-4369-9060-c367302aacad None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:42.493575 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916d8895-bd92-4369-9060-c367302aacad None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:42.494447 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916d8895-bd92-4369-9060-c367302aacad None admin] RBAC: Adding query filter params (name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:42.494929 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916d8895-bd92-4369-9060-c367302aacad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:42.497153 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916d8895-bd92-4369-9060-c367302aacad None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:42.512567 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 274/545] 10.222.0.26 () {56 vars in 1103 bytes} [Sat Aug 6 01:20:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:20:42.518110 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e632bd35-e5a8-434f-95be-adf07fadf7e3 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:42.631897 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e632bd35-e5a8-434f-95be-adf07fadf7e3 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:42.676972 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b6b832-6479-4117-a21f-53cc6485c4e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:42.677592 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b6b832-6479-4117-a21f-53cc6485c4e3 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:42.677876 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b6b832-6479-4117-a21f-53cc6485c4e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:42.678153 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b6b832-6479-4117-a21f-53cc6485c4e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:42.691554 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b6b832-6479-4117-a21f-53cc6485c4e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:42.692189 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b6b832-6479-4117-a21f-53cc6485c4e3 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:42.712561 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-56b6b832-6479-4117-a21f-53cc6485c4e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:20:42.712561 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:42.712561 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:42.712561 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:42.712561 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:42.712561 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:42.712561 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:42.712561 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:42.712561 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:42.712561 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:42.712561 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:42.712561 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:42.712561 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:20:42.712561 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:20:42.712561 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:20:42.713480 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:20:42.713480 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:42.713480 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:42.713480 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:42.713480 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:42.713480 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:42.713480 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:42.713480 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:42.713480 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:42.713480 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:42.713480 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:42.713480 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:42.713480 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:42.713480 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:42.713480 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:42.713480 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:42.713480 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:42.714338 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:20:42.714338 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:20:42.714338 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:20:42.714338 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:20:42.718178 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 272/546] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:20:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:20:42.723718 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916d8895-bd92-4369-9060-c367302aacad None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:42.892118 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916d8895-bd92-4369-9060-c367302aacad None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:42.929013 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba027e0-7f59-4986-a7bb-8ad958f5bb12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:42.929617 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba027e0-7f59-4986-a7bb-8ad958f5bb12 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:42.929900 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba027e0-7f59-4986-a7bb-8ad958f5bb12 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:42.930176 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba027e0-7f59-4986-a7bb-8ad958f5bb12 None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:42.930867 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba027e0-7f59-4986-a7bb-8ad958f5bb12 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:42.931174 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba027e0-7f59-4986-a7bb-8ad958f5bb12 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:42.932544 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba027e0-7f59-4986-a7bb-8ad958f5bb12 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:42.947977 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 275/547] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Aug 6 01:20:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:42.951413 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b6b832-6479-4117-a21f-53cc6485c4e3 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:42.971253 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7c3d65-e0ae-402a-8527-b411caac7e7b None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:42.971538 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7c3d65-e0ae-402a-8527-b411caac7e7b None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:42.971815 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7c3d65-e0ae-402a-8527-b411caac7e7b None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:42.973435 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 276/548] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:20:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 01:20:42.978908 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56070ff-b7a1-498c-86ae-b09ae0dd6074 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:42.979197 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56070ff-b7a1-498c-86ae-b09ae0dd6074 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:42.979472 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56070ff-b7a1-498c-86ae-b09ae0dd6074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:43.013478 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-f56070ff-b7a1-498c-86ae-b09ae0dd6074 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:20:43.014685 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-f56070ff-b7a1-498c-86ae-b09ae0dd6074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:20:43.030119 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:43.087084 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b6b832-6479-4117-a21f-53cc6485c4e3 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:43.117494 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:43.129074 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14aff0b9-3882-4045-bce3-f360595396f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:43.129074 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aff0b9-3882-4045-bce3-f360595396f5 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:43.129074 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aff0b9-3882-4045-bce3-f360595396f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:43.129074 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aff0b9-3882-4045-bce3-f360595396f5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:43.150331 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14aff0b9-3882-4045-bce3-f360595396f5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:43.152881 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14aff0b9-3882-4045-bce3-f360595396f5 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:43.166708 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 277/549] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:20:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:20:43.169638 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-14aff0b9-3882-4045-bce3-f360595396f5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 06 01:20:43.169638 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:43.169638 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:43.169638 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:43.169638 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:43.169638 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:43.169638 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:43.169638 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:43.169638 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:43.169638 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:43.169638 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:43.169638 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:43.169638 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 01:20:43.169638 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 01:20:43.169638 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 01:20:43.169638 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 01:20:43.170975 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:43.170975 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:43.170975 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 01:20:43.170975 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:20:43.170975 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 01:20:43.170975 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:20:43.170975 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:43.170975 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:43.170975 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:43.170975 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:43.170975 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:43.170975 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:43.170975 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:43.170975 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:43.170975 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:43.170975 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:43.170975 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:43.170975 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:43.172124 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:43.172124 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:43.172124 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 01:20:43.172124 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 01:20:43.172124 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 01:20:43.172124 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 01:20:43.172124 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 01:20:43.172124 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 01:20:43.172124 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 06 01:20:43.172124 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:20:43.172124 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 273/550] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Aug 6 01:20:42 2022] GET /identity/v3/users/swift => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:20:43.176993 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56070ff-b7a1-498c-86ae-b09ae0dd6074 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:43.204886 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14aff0b9-3882-4045-bce3-f360595396f5 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:43.384684 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56070ff-b7a1-498c-86ae-b09ae0dd6074 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:43.444515 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14aff0b9-3882-4045-bce3-f360595396f5 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:43.447145 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd6c586-0fb5-4bee-95af-7feb7eb6d211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:43.447781 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd6c586-0fb5-4bee-95af-7feb7eb6d211 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:43.448062 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd6c586-0fb5-4bee-95af-7feb7eb6d211 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:43.448337 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd6c586-0fb5-4bee-95af-7feb7eb6d211 None admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:43.449369 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd6c586-0fb5-4bee-95af-7feb7eb6d211 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:43.449795 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd6c586-0fb5-4bee-95af-7feb7eb6d211 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:43.451250 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd6c586-0fb5-4bee-95af-7feb7eb6d211 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:43.461491 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:43.486125 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 278/551] 10.222.0.26 () {56 vars in 1135 bytes} [Sat Aug 6 01:20:43 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:43.491348 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd6c586-0fb5-4bee-95af-7feb7eb6d211 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:43.510617 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db51728-3092-4187-88f8-bbf40b3a6325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:43.511289 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db51728-3092-4187-88f8-bbf40b3a6325 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:43.511649 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db51728-3092-4187-88f8-bbf40b3a6325 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:43.512009 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db51728-3092-4187-88f8-bbf40b3a6325 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:43.531865 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db51728-3092-4187-88f8-bbf40b3a6325 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:20:43.538213 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-1db51728-3092-4187-88f8-bbf40b3a6325 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 06 01:20:43.538213 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:43.538213 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:43.538213 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:43.538213 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:43.538213 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:43.538213 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:43.538213 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:43.538213 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:43.538213 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:43.538213 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:43.538213 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:43.538213 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:20:43.538213 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:20:43.538213 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:20:43.538213 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:20:43.539127 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:20:43.539127 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:20:43.539127 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:20:43.539127 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:20:43.539127 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:43.539127 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:43.539127 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:43.539127 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:43.539127 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:43.539127 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:43.539127 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:43.539127 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:43.539127 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:43.539127 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:43.539127 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:43.539127 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:43.540105 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:43.540105 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:43.540105 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:43.540105 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:43.540105 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:20:43.540105 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:20:43.540105 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:20:43.540105 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:20:43.540105 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:20:43.540105 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:20:43.540105 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 06 01:20:43.540105 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:20:43.540105 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 274/552] 10.222.0.26 () {56 vars in 1102 bytes} [Sat Aug 6 01:20:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:20:43.545987 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db51728-3092-4187-88f8-bbf40b3a6325 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:43.580829 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:43.660421 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db51728-3092-4187-88f8-bbf40b3a6325 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:43.697132 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd6c586-0fb5-4bee-95af-7feb7eb6d211 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:43.722426 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92df6b51-3f7a-4968-9867-4f8bc39726ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:43.723886 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92df6b51-3f7a-4968-9867-4f8bc39726ed None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:43.724388 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92df6b51-3f7a-4968-9867-4f8bc39726ed None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:43.724915 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92df6b51-3f7a-4968-9867-4f8bc39726ed None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:43.727744 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92df6b51-3f7a-4968-9867-4f8bc39726ed None admin] RBAC: Adding query filter params (name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:43.728267 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92df6b51-3f7a-4968-9867-4f8bc39726ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:43.729878 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92df6b51-3f7a-4968-9867-4f8bc39726ed None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:43.736356 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bf4578-c0f2-48a7-b40d-25a3bb3d8312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:43.736356 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bf4578-c0f2-48a7-b40d-25a3bb3d8312 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:43.736356 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bf4578-c0f2-48a7-b40d-25a3bb3d8312 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:43.736356 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bf4578-c0f2-48a7-b40d-25a3bb3d8312 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:43.744883 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 275/553] 10.222.0.26 () {56 vars in 1103 bytes} [Sat Aug 6 01:20:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:43.750546 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92df6b51-3f7a-4968-9867-4f8bc39726ed None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:43.802867 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:43.814144 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bf4578-c0f2-48a7-b40d-25a3bb3d8312 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:43.815807 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bf4578-c0f2-48a7-b40d-25a3bb3d8312 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:43.851841 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:43.892664 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92df6b51-3f7a-4968-9867-4f8bc39726ed None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:43.927211 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 279/554] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Aug 6 01:20:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 06 01:20:43.928833 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849714ce-83a7-4628-a28a-d44ec5d74129 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:43.929342 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849714ce-83a7-4628-a28a-d44ec5d74129 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:43.929783 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849714ce-83a7-4628-a28a-d44ec5d74129 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:43.931550 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 280/555] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:20:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:20:43.932693 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849714ce-83a7-4628-a28a-d44ec5d74129 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:43.938290 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8610bf2-8dc6-45c9-a875-fafbcc942535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:43.938964 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8610bf2-8dc6-45c9-a875-fafbcc942535 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:43.939458 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8610bf2-8dc6-45c9-a875-fafbcc942535 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:43.939929 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8610bf2-8dc6-45c9-a875-fafbcc942535 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:43.953002 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8610bf2-8dc6-45c9-a875-fafbcc942535 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:43.954694 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8610bf2-8dc6-45c9-a875-fafbcc942535 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:43.971004 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-d8610bf2-8dc6-45c9-a875-fafbcc942535 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:20:43.971004 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:43.971004 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:43.971004 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:43.971004 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:43.971004 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:43.971004 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:43.971004 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:43.971004 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:43.971004 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:43.971004 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:43.971004 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:43.971004 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:20:43.971004 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:20:43.971004 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:20:43.971905 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:20:43.971905 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:43.971905 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:43.971905 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:43.971905 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:43.971905 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:43.971905 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:43.971905 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:43.971905 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:43.971905 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:43.971905 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:43.971905 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:43.971905 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:43.971905 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:43.971905 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:43.971905 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:43.971905 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:43.972817 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:20:43.972817 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:20:43.972817 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:20:43.972817 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:20:43.973533 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 276/556] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:20:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:20:43.974666 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7095b05a-44b0-4833-a0b9-1bd3408b12d0 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:43.974947 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7095b05a-44b0-4833-a0b9-1bd3408b12d0 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:43.975222 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7095b05a-44b0-4833-a0b9-1bd3408b12d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:43.995008 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-7095b05a-44b0-4833-a0b9-1bd3408b12d0 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:20:43.996778 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-7095b05a-44b0-4833-a0b9-1bd3408b12d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:20:44.050271 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849714ce-83a7-4628-a28a-d44ec5d74129 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:44.074047 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 277/557] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:20:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:20:44.074910 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7095b05a-44b0-4833-a0b9-1bd3408b12d0 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:44.087398 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:44.094082 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493f7d39-a1a4-4d3c-8393-d09d3bc13650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:44.095099 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493f7d39-a1a4-4d3c-8393-d09d3bc13650 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:44.095704 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493f7d39-a1a4-4d3c-8393-d09d3bc13650 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:44.096178 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493f7d39-a1a4-4d3c-8393-d09d3bc13650 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:44.098432 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:44.109531 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493f7d39-a1a4-4d3c-8393-d09d3bc13650 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:44.111358 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493f7d39-a1a4-4d3c-8393-d09d3bc13650 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:44.127159 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-493f7d39-a1a4-4d3c-8393-d09d3bc13650 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:20:44.127159 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:44.127159 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:44.127159 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:44.127159 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:44.127159 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:44.127159 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:44.127159 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:44.127159 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:44.127159 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:44.127159 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:44.127159 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:44.127159 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:20:44.127159 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:20:44.127159 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:20:44.128551 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:20:44.128551 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:44.128551 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:44.128551 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:44.128551 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:44.128551 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:44.128551 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:44.128551 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:44.128551 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:44.128551 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:44.128551 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:44.128551 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:44.128551 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:44.128551 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:44.128551 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:44.128551 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:44.128551 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:44.129463 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:20:44.129463 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:20:44.129463 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:20:44.129463 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:20:44.131740 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 281/558] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:20:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:20:44.132765 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493f7d39-a1a4-4d3c-8393-d09d3bc13650 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:44.195072 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7095b05a-44b0-4833-a0b9-1bd3408b12d0 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:44.236075 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595932d2-7b5a-4c9e-9221-f624dcd73d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:44.236642 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595932d2-7b5a-4c9e-9221-f624dcd73d19 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:44.236974 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595932d2-7b5a-4c9e-9221-f624dcd73d19 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:44.237295 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595932d2-7b5a-4c9e-9221-f624dcd73d19 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:44.238149 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595932d2-7b5a-4c9e-9221-f624dcd73d19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:44.238457 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595932d2-7b5a-4c9e-9221-f624dcd73d19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:44.240547 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595932d2-7b5a-4c9e-9221-f624dcd73d19 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:44.244625 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493f7d39-a1a4-4d3c-8393-d09d3bc13650 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:44.253529 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 278/559] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Aug 6 01:20:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:44.254490 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595932d2-7b5a-4c9e-9221-f624dcd73d19 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:44.283427 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6389764a-5f66-41b7-b1d6-ef20e32c680f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:44.284419 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6389764a-5f66-41b7-b1d6-ef20e32c680f None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:44.285042 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6389764a-5f66-41b7-b1d6-ef20e32c680f None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:44.285667 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6389764a-5f66-41b7-b1d6-ef20e32c680f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:44.297582 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6389764a-5f66-41b7-b1d6-ef20e32c680f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:20:44.303437 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-6389764a-5f66-41b7-b1d6-ef20e32c680f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 06 01:20:44.303437 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:44.303437 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:44.303437 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:44.303437 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:44.303437 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:44.303437 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:44.303437 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:44.303437 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:44.303437 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:44.303437 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:44.303437 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:44.303437 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:20:44.303437 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:20:44.303437 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:20:44.303437 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:20:44.304360 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:20:44.304360 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:20:44.304360 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:20:44.304360 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:20:44.304360 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:44.304360 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:44.304360 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:44.304360 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:44.304360 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:44.304360 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:44.304360 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:44.304360 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:44.304360 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:44.304360 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:44.304360 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:44.304360 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:44.305320 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:44.305320 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:44.305320 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:44.305320 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:44.305320 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:20:44.305320 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:20:44.305320 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:20:44.305320 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:20:44.305320 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:20:44.305320 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:20:44.305320 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 06 01:20:44.305320 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:20:44.308054 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 282/560] 10.222.0.26 () {56 vars in 1102 bytes} [Sat Aug 6 01:20:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:20:44.310178 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6389764a-5f66-41b7-b1d6-ef20e32c680f None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:44.335977 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:44.362738 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:44.369867 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595932d2-7b5a-4c9e-9221-f624dcd73d19 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:44.411000 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e9fd21-6896-4ba6-8dee-06b0f45b0bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:44.412016 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e9fd21-6896-4ba6-8dee-06b0f45b0bf7 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:44.412357 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e9fd21-6896-4ba6-8dee-06b0f45b0bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:44.412696 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e9fd21-6896-4ba6-8dee-06b0f45b0bf7 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:44.413493 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e9fd21-6896-4ba6-8dee-06b0f45b0bf7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:44.413852 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e9fd21-6896-4ba6-8dee-06b0f45b0bf7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:44.415218 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e9fd21-6896-4ba6-8dee-06b0f45b0bf7 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:44.425704 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6389764a-5f66-41b7-b1d6-ef20e32c680f None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:44.432049 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 279/561] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Aug 6 01:20:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:44.432049 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e9fd21-6896-4ba6-8dee-06b0f45b0bf7 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:44.466860 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ae50a2-1ab7-48a1-b448-e9cf8f759709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:44.467514 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ae50a2-1ab7-48a1-b448-e9cf8f759709 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:44.467798 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ae50a2-1ab7-48a1-b448-e9cf8f759709 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:44.468076 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ae50a2-1ab7-48a1-b448-e9cf8f759709 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:44.476174 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ae50a2-1ab7-48a1-b448-e9cf8f759709 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:44.477666 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ae50a2-1ab7-48a1-b448-e9cf8f759709 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:44.489779 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-b7ae50a2-1ab7-48a1-b448-e9cf8f759709 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 06 01:20:44.489779 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:44.489779 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:44.489779 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:44.489779 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:44.489779 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:44.489779 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:44.489779 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:44.489779 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:44.489779 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:44.489779 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:44.489779 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:44.489779 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 01:20:44.489779 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 01:20:44.489779 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 01:20:44.489779 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 01:20:44.490814 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:44.490814 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:44.490814 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 01:20:44.490814 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:20:44.490814 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 01:20:44.490814 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:20:44.490814 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:44.490814 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:44.490814 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:44.490814 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:44.490814 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:44.490814 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:44.490814 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:44.490814 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:44.490814 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:44.490814 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:44.490814 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:44.490814 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:44.491746 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:44.491746 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:44.491746 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 01:20:44.491746 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 01:20:44.491746 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 01:20:44.491746 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 01:20:44.491746 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 01:20:44.491746 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 01:20:44.491746 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 06 01:20:44.491746 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:20:44.491746 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 283/562] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Aug 6 01:20:44 2022] GET /identity/v3/users/glance => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:20:44.492321 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e622fd1f-c07a-4c45-adc4-cf30b24258ed None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:44.493092 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e622fd1f-c07a-4c45-adc4-cf30b24258ed None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:44.493092 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e622fd1f-c07a-4c45-adc4-cf30b24258ed None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:44.497194 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 284/563] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:20:44 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:20:44.497194 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e622fd1f-c07a-4c45-adc4-cf30b24258ed None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:44.547877 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e9fd21-6896-4ba6-8dee-06b0f45b0bf7 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:44.579703 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:44.587855 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c68372-0e25-4b2e-aa6e-d24268f88a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:44.588496 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c68372-0e25-4b2e-aa6e-d24268f88a37 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:44.588815 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c68372-0e25-4b2e-aa6e-d24268f88a37 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:44.589153 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c68372-0e25-4b2e-aa6e-d24268f88a37 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:44.590444 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c68372-0e25-4b2e-aa6e-d24268f88a37 None admin] RBAC: Adding query filter params (name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:44.590777 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c68372-0e25-4b2e-aa6e-d24268f88a37 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:44.592171 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c68372-0e25-4b2e-aa6e-d24268f88a37 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:44.600188 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 280/564] 10.222.0.26 () {56 vars in 1103 bytes} [Sat Aug 6 01:20:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:44.601287 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c68372-0e25-4b2e-aa6e-d24268f88a37 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:44.613965 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e622fd1f-c07a-4c45-adc4-cf30b24258ed None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:44.622050 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:44.656018 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e42df85-ce06-40b4-b309-7a4bbca0f107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:44.656809 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e42df85-ce06-40b4-b309-7a4bbca0f107 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:44.657240 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e42df85-ce06-40b4-b309-7a4bbca0f107 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:44.657613 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e42df85-ce06-40b4-b309-7a4bbca0f107 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:44.664256 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e42df85-ce06-40b4-b309-7a4bbca0f107 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:44.665714 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e42df85-ce06-40b4-b309-7a4bbca0f107 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:44.677315 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-7e42df85-ce06-40b4-b309-7a4bbca0f107 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:20:44.677315 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:44.677315 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:44.677315 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:44.677315 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:44.677315 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:44.677315 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:44.677315 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:44.677315 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:44.677315 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:44.677315 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:44.677315 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:44.677315 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 01:20:44.677315 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 01:20:44.677315 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 01:20:44.678220 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 01:20:44.678220 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:44.678220 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:44.678220 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:44.678220 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:44.678220 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:44.678220 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:44.678220 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:44.678220 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:44.678220 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:44.678220 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:44.678220 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:44.678220 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:44.678220 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:44.678220 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:44.678220 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:44.678220 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:44.679099 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 01:20:44.679099 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 01:20:44.679099 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 01:20:44.679099 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 01:20:44.679099 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 01:20:44.679099 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 01:20:44.679099 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:20:44.679099 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:20:44.679099 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 285/565] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Aug 6 01:20:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:20:44.679575 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e55b356-54c1-49a0-ba88-2cc07e130e7c None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:44.680277 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e55b356-54c1-49a0-ba88-2cc07e130e7c None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:44.680277 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e55b356-54c1-49a0-ba88-2cc07e130e7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:44.698440 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-9e55b356-54c1-49a0-ba88-2cc07e130e7c None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:20:44.700187 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-9e55b356-54c1-49a0-ba88-2cc07e130e7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:20:44.719547 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c68372-0e25-4b2e-aa6e-d24268f88a37 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:44.755707 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ce6f95-729d-435e-bb12-00f46ab8edd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:44.756337 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ce6f95-729d-435e-bb12-00f46ab8edd6 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:44.756975 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ce6f95-729d-435e-bb12-00f46ab8edd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:44.756975 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ce6f95-729d-435e-bb12-00f46ab8edd6 None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:44.757614 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ce6f95-729d-435e-bb12-00f46ab8edd6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:44.757933 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ce6f95-729d-435e-bb12-00f46ab8edd6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:44.759358 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ce6f95-729d-435e-bb12-00f46ab8edd6 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:44.771833 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 286/566] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:20:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:20:44.773320 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e55b356-54c1-49a0-ba88-2cc07e130e7c None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:44.781111 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 281/567] 10.222.0.26 () {56 vars in 1137 bytes} [Sat Aug 6 01:20:44 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:44.781111 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ce6f95-729d-435e-bb12-00f46ab8edd6 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:44.835112 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:44.882024 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:44.888252 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e55b356-54c1-49a0-ba88-2cc07e130e7c None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:44.888988 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ce6f95-729d-435e-bb12-00f46ab8edd6 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:44.924214 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ca3282-f169-4141-9542-11dccd867095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:44.924425 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd74c22-1013-45bb-9c4b-113b66467995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:44.924837 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ca3282-f169-4141-9542-11dccd867095 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:44.925060 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd74c22-1013-45bb-9c4b-113b66467995 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:44.925169 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ca3282-f169-4141-9542-11dccd867095 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:44.925350 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd74c22-1013-45bb-9c4b-113b66467995 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:44.925440 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ca3282-f169-4141-9542-11dccd867095 None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:44.925624 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd74c22-1013-45bb-9c4b-113b66467995 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:44.926154 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ca3282-f169-4141-9542-11dccd867095 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:44.926467 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ca3282-f169-4141-9542-11dccd867095 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:44.927901 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ca3282-f169-4141-9542-11dccd867095 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:44.936316 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd74c22-1013-45bb-9c4b-113b66467995 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:44.937775 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd74c22-1013-45bb-9c4b-113b66467995 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:44.939852 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 282/568] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Aug 6 01:20:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:44.940644 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ca3282-f169-4141-9542-11dccd867095 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:44.957347 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-6cd74c22-1013-45bb-9c4b-113b66467995 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:20:44.957347 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:44.957347 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:44.957347 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:44.957347 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:44.957347 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:44.957347 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:44.957347 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:44.957347 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:44.957347 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:44.957347 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:44.957347 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:44.957347 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:20:44.957347 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:20:44.957347 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:20:44.958284 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:20:44.958284 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:44.958284 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:44.958284 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:44.958284 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:44.958284 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:44.958284 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:44.958284 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:44.958284 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:44.958284 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:44.958284 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:44.958284 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:44.958284 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:44.958284 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:44.958284 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:44.958284 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:44.958284 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:44.959149 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:20:44.959149 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:20:44.959149 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:20:44.959149 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:20:44.959149 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 287/569] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:20:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:20:44.959537 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd74c22-1013-45bb-9c4b-113b66467995 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:45.046472 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ca3282-f169-4141-9542-11dccd867095 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:45.059851 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:45.062755 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd74c22-1013-45bb-9c4b-113b66467995 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:45.084751 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2678d8-29ee-4372-b3d6-fa47a81d68c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:45.085424 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2678d8-29ee-4372-b3d6-fa47a81d68c9 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:45.085717 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2678d8-29ee-4372-b3d6-fa47a81d68c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:45.085992 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2678d8-29ee-4372-b3d6-fa47a81d68c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:45.095507 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bc5ab7-7f32-4803-8bbf-2fb981b97ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:45.096135 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bc5ab7-7f32-4803-8bbf-2fb981b97ff9 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:45.096413 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bc5ab7-7f32-4803-8bbf-2fb981b97ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:45.096505 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2678d8-29ee-4372-b3d6-fa47a81d68c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:45.096684 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bc5ab7-7f32-4803-8bbf-2fb981b97ff9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:45.097962 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2678d8-29ee-4372-b3d6-fa47a81d68c9 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:45.107865 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bc5ab7-7f32-4803-8bbf-2fb981b97ff9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:20:45.113112 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-3e2678d8-29ee-4372-b3d6-fa47a81d68c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:20:45.113112 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:45.113112 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:45.113112 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:45.113112 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:45.113112 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:45.113112 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:45.113112 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:45.113112 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:45.113112 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:45.113112 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:45.113112 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:45.113112 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:20:45.113112 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:20:45.113112 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:20:45.114011 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:20:45.114011 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:45.114011 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:45.114011 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:45.114011 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:45.114011 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:45.114011 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:45.114011 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:45.114011 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:45.114011 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:45.114011 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:45.114011 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:45.114011 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:45.114011 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:45.114011 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:45.114011 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:45.114011 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:45.114878 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:20:45.114878 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:20:45.114878 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:20:45.114878 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:20:45.115092 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-f9bc5ab7-7f32-4803-8bbf-2fb981b97ff9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 06 01:20:45.115092 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:45.115092 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:45.115092 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:45.115092 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:45.115092 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:45.115092 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:45.115092 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:45.115092 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:45.115092 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:45.115092 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:45.115092 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:45.115092 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:20:45.115092 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:20:45.115092 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:20:45.115092 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:20:45.115913 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:20:45.115913 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:20:45.115913 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:20:45.115913 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:20:45.115913 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:45.115913 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:45.115913 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:45.115913 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:45.115913 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:45.115913 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:45.115913 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:45.115913 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:45.115913 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:45.115913 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:45.115913 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:45.115913 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:45.116832 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:45.116832 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:45.116832 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:45.116832 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:45.116832 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:20:45.116832 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:20:45.116832 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:20:45.116832 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:20:45.116832 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:20:45.116832 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:20:45.116832 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 06 01:20:45.116832 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:20:45.116832 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 288/570] 10.222.0.26 () {56 vars in 1102 bytes} [Sat Aug 6 01:20:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:20:45.117557 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 283/571] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:20:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:20:45.117557 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2678d8-29ee-4372-b3d6-fa47a81d68c9 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:45.119880 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65df8011-a8f6-4213-96b7-dced194a3a62 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:45.120165 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65df8011-a8f6-4213-96b7-dced194a3a62 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:45.120445 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65df8011-a8f6-4213-96b7-dced194a3a62 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:45.123703 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 289/572] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:20:45 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:20:45.123703 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65df8011-a8f6-4213-96b7-dced194a3a62 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:45.134419 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:45.226112 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2678d8-29ee-4372-b3d6-fa47a81d68c9 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:45.234521 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65df8011-a8f6-4213-96b7-dced194a3a62 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:45.265700 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb615999-f956-4461-af4e-e2792aca56db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:45.266409 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb615999-f956-4461-af4e-e2792aca56db None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:45.266691 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb615999-f956-4461-af4e-e2792aca56db None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:45.266991 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb615999-f956-4461-af4e-e2792aca56db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:45.267783 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb615999-f956-4461-af4e-e2792aca56db None admin] RBAC: Adding query filter params (role.id=c9ce798acd4b482f8f391c6af97eb7a4, scope.project.id=d9d68d310bf24449a5fe20d2dc264a58, user.id=2b23f4478f49471fba6f43e6e4be99da) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:45.268328 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb615999-f956-4461-af4e-e2792aca56db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:45.269595 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb615999-f956-4461-af4e-e2792aca56db None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:45.269898 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bea52c-c3dd-4be9-83a5-208cbcf997cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:45.270530 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bea52c-c3dd-4be9-83a5-208cbcf997cf None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:45.270817 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bea52c-c3dd-4be9-83a5-208cbcf997cf None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:45.271098 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bea52c-c3dd-4be9-83a5-208cbcf997cf None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:45.271839 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bea52c-c3dd-4be9-83a5-208cbcf997cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:45.272178 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bea52c-c3dd-4be9-83a5-208cbcf997cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:45.273595 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bea52c-c3dd-4be9-83a5-208cbcf997cf None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:45.275296 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 284/573] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Aug 6 01:20:45 2022] GET /identity/v3/role_assignments?user.id=2b23f4478f49471fba6f43e6e4be99da&scope.project.id=d9d68d310bf24449a5fe20d2dc264a58&role.id=c9ce798acd4b482f8f391c6af97eb7a4 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:45.276106 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb615999-f956-4461-af4e-e2792aca56db None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:45.301659 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 290/574] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Aug 6 01:20:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:45.303634 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bea52c-c3dd-4be9-83a5-208cbcf997cf None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:45.314990 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:45.387280 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:45.436010 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bea52c-c3dd-4be9-83a5-208cbcf997cf None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:45.436205 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb615999-f956-4461-af4e-e2792aca56db None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:45.477661 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d30125-0d04-4335-8472-106de6913882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:45.478286 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d30125-0d04-4335-8472-106de6913882 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:45.478565 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d30125-0d04-4335-8472-106de6913882 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:45.478840 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d30125-0d04-4335-8472-106de6913882 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:45.479558 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d30125-0d04-4335-8472-106de6913882 None admin] RBAC: Adding query filter params (name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:45.479862 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d30125-0d04-4335-8472-106de6913882 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:45.480742 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b35879-dec5-4bfd-9a88-9a227dd6f8e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:45.481579 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b35879-dec5-4bfd-9a88-9a227dd6f8e4 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:45.481864 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b35879-dec5-4bfd-9a88-9a227dd6f8e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:45.481973 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d30125-0d04-4335-8472-106de6913882 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:45.482151 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b35879-dec5-4bfd-9a88-9a227dd6f8e4 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:45.482857 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b35879-dec5-4bfd-9a88-9a227dd6f8e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:45.483167 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b35879-dec5-4bfd-9a88-9a227dd6f8e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:45.484543 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b35879-dec5-4bfd-9a88-9a227dd6f8e4 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:45.490851 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 291/575] 10.222.0.26 () {56 vars in 1103 bytes} [Sat Aug 6 01:20:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:45.492332 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94152bf-c7dc-4152-b674-530caa738bcd None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:45.492816 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94152bf-c7dc-4152-b674-530caa738bcd None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:45.493311 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94152bf-c7dc-4152-b674-530caa738bcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:45.501783 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 285/576] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Aug 6 01:20:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:45.502909 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b35879-dec5-4bfd-9a88-9a227dd6f8e4 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:45.513444 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-c94152bf-c7dc-4152-b674-530caa738bcd None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:20:45.515121 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-c94152bf-c7dc-4152-b674-530caa738bcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:20:45.592795 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 292/577] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:20:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:20:45.594459 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94152bf-c7dc-4152-b674-530caa738bcd None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:45.606931 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:45.627064 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b35879-dec5-4bfd-9a88-9a227dd6f8e4 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:45.663230 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45b4953-b447-41f2-8502-3ee8dc139609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:45.663876 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45b4953-b447-41f2-8502-3ee8dc139609 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:45.664590 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45b4953-b447-41f2-8502-3ee8dc139609 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:45.664590 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45b4953-b447-41f2-8502-3ee8dc139609 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:45.674440 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:45.675812 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45b4953-b447-41f2-8502-3ee8dc139609 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:45.677458 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45b4953-b447-41f2-8502-3ee8dc139609 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:45.691406 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-d45b4953-b447-41f2-8502-3ee8dc139609 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 06 01:20:45.691406 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:45.691406 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:45.691406 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:45.691406 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:45.691406 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:45.691406 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:45.691406 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:45.691406 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:45.691406 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:45.691406 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:45.691406 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:45.691406 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 01:20:45.691406 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 01:20:45.691406 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 01:20:45.691406 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 01:20:45.692855 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:45.692855 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:45.692855 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 01:20:45.692855 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:20:45.692855 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 01:20:45.692855 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:20:45.692855 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:45.692855 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:45.692855 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:45.692855 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:45.692855 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:45.692855 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:45.692855 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:45.692855 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:45.692855 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:45.692855 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:45.692855 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:45.692855 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:45.694413 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:45.694413 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:45.694413 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 01:20:45.694413 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 01:20:45.694413 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 01:20:45.694413 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 01:20:45.694413 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 01:20:45.694413 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 01:20:45.694413 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 06 01:20:45.694413 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:20:45.694413 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 286/578] 10.222.0.26 () {56 vars in 1093 bytes} [Sat Aug 6 01:20:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:20:45.694413 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45b4953-b447-41f2-8502-3ee8dc139609 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:45.746724 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94152bf-c7dc-4152-b674-530caa738bcd None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:45.785261 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d5cba0-9215-4ebe-8c05-ba31e6a6b329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:45.786175 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d5cba0-9215-4ebe-8c05-ba31e6a6b329 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:45.786175 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d5cba0-9215-4ebe-8c05-ba31e6a6b329 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:45.786449 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d5cba0-9215-4ebe-8c05-ba31e6a6b329 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:45.801309 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d5cba0-9215-4ebe-8c05-ba31e6a6b329 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:45.803784 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d5cba0-9215-4ebe-8c05-ba31e6a6b329 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:45.821572 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-e6d5cba0-9215-4ebe-8c05-ba31e6a6b329 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:20:45.821572 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:45.821572 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:45.821572 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:45.821572 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:45.821572 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:45.821572 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:45.821572 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:45.821572 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:45.821572 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:45.821572 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:45.821572 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:45.821572 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:20:45.821572 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:20:45.821572 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:20:45.822704 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:20:45.822704 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:45.822704 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:45.822704 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:45.822704 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:45.822704 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:45.822704 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:45.822704 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:45.822704 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:45.822704 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:45.822704 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:45.822704 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:45.822704 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:45.822704 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:45.822704 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:45.822704 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:45.822704 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:45.823587 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:20:45.823587 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:20:45.823587 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:20:45.823587 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:20:45.823587 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 293/579] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:20:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:20:45.824400 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d5cba0-9215-4ebe-8c05-ba31e6a6b329 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:45.836325 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45b4953-b447-41f2-8502-3ee8dc139609 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:45.882281 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b674c5-9bf3-4e86-9b40-3a264b98573a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:45.883749 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b674c5-9bf3-4e86-9b40-3a264b98573a None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:45.884833 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b674c5-9bf3-4e86-9b40-3a264b98573a None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:45.884833 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b674c5-9bf3-4e86-9b40-3a264b98573a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:45.893548 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b674c5-9bf3-4e86-9b40-3a264b98573a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:45.895002 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b674c5-9bf3-4e86-9b40-3a264b98573a None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:45.908746 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-e8b674c5-9bf3-4e86-9b40-3a264b98573a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:20:45.908746 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:45.908746 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:45.908746 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:45.908746 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:45.908746 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:45.908746 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:45.908746 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:45.908746 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:45.908746 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:45.908746 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:45.908746 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:45.908746 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 01:20:45.908746 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 01:20:45.908746 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 01:20:45.909723 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 01:20:45.909723 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:45.909723 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:45.909723 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:45.909723 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:45.909723 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:45.909723 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:45.909723 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:45.909723 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:45.909723 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:45.909723 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:45.909723 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:45.909723 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:45.909723 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:45.909723 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:45.909723 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:45.909723 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:45.910606 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 01:20:45.910606 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 01:20:45.910606 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 01:20:45.910606 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 01:20:45.910606 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 01:20:45.910606 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 01:20:45.910606 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:20:45.910606 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:20:45.913376 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 287/580] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Aug 6 01:20:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:20:45.914230 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b674c5-9bf3-4e86-9b40-3a264b98573a None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:45.924496 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:45.958000 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d5cba0-9215-4ebe-8c05-ba31e6a6b329 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:45.972570 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:45.998642 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d440b66-e674-4c0e-bc30-89815aa0527e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:45.999388 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d440b66-e674-4c0e-bc30-89815aa0527e None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:45.999670 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d440b66-e674-4c0e-bc30-89815aa0527e None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:45.999941 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d440b66-e674-4c0e-bc30-89815aa0527e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:46.010549 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d440b66-e674-4c0e-bc30-89815aa0527e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:20:46.015911 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-7d440b66-e674-4c0e-bc30-89815aa0527e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 06 01:20:46.015911 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:46.015911 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:46.015911 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:46.015911 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:46.015911 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:46.015911 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:46.015911 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:46.015911 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:46.015911 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:46.015911 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:46.015911 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:46.015911 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:20:46.015911 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:20:46.015911 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:20:46.015911 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:20:46.017039 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:20:46.017039 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:20:46.017039 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:20:46.017039 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:20:46.017039 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:46.017039 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:46.017039 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:46.017039 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:46.017039 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:46.017039 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:46.017039 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:46.017039 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:46.017039 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:46.017039 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:46.017039 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:46.017039 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:46.017920 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:46.017920 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:46.017920 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:46.017920 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:46.017920 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:20:46.017920 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:20:46.017920 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:20:46.017920 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:20:46.017920 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:20:46.017920 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:20:46.017920 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 06 01:20:46.017920 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:20:46.017920 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 294/581] 10.222.0.26 () {56 vars in 1102 bytes} [Sat Aug 6 01:20:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:20:46.017920 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d440b66-e674-4c0e-bc30-89815aa0527e None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:46.031622 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b674c5-9bf3-4e86-9b40-3a264b98573a None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:46.069570 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bdd074-a348-4900-8559-3920648b0cc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:46.070658 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bdd074-a348-4900-8559-3920648b0cc7 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:46.070946 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bdd074-a348-4900-8559-3920648b0cc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:46.071232 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bdd074-a348-4900-8559-3920648b0cc7 None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:46.072436 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bdd074-a348-4900-8559-3920648b0cc7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:46.072754 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bdd074-a348-4900-8559-3920648b0cc7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:46.074269 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bdd074-a348-4900-8559-3920648b0cc7 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:46.093707 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 288/582] 10.222.0.26 () {56 vars in 1133 bytes} [Sat Aug 6 01:20:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:46.094795 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bdd074-a348-4900-8559-3920648b0cc7 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:46.126144 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d440b66-e674-4c0e-bc30-89815aa0527e None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:46.161755 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b842d77e-37b4-4e17-9494-548cf636e8d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:46.162291 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b842d77e-37b4-4e17-9494-548cf636e8d9 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:46.162573 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b842d77e-37b4-4e17-9494-548cf636e8d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:46.162848 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b842d77e-37b4-4e17-9494-548cf636e8d9 None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:46.162958 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:46.163765 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b842d77e-37b4-4e17-9494-548cf636e8d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:46.163889 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b842d77e-37b4-4e17-9494-548cf636e8d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:46.165534 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b842d77e-37b4-4e17-9494-548cf636e8d9 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:46.182540 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 295/583] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Aug 6 01:20:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:46.182540 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b842d77e-37b4-4e17-9494-548cf636e8d9 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:46.205499 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bdd074-a348-4900-8559-3920648b0cc7 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:46.219640 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:46.245940 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84131c3-3ace-4a6d-bf2c-994a9785f15e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:46.246874 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84131c3-3ace-4a6d-bf2c-994a9785f15e None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:46.247166 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84131c3-3ace-4a6d-bf2c-994a9785f15e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:46.247613 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84131c3-3ace-4a6d-bf2c-994a9785f15e None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:46.248709 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84131c3-3ace-4a6d-bf2c-994a9785f15e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:46.249204 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84131c3-3ace-4a6d-bf2c-994a9785f15e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:46.250663 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84131c3-3ace-4a6d-bf2c-994a9785f15e None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:46.263106 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 289/584] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Aug 6 01:20:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:46.263693 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84131c3-3ace-4a6d-bf2c-994a9785f15e None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:46.300135 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b842d77e-37b4-4e17-9494-548cf636e8d9 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:46.335953 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cba3594-bf6a-4e7a-ad45-8450617e202a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:46.336573 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cba3594-bf6a-4e7a-ad45-8450617e202a None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:46.336855 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cba3594-bf6a-4e7a-ad45-8450617e202a None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:46.337157 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cba3594-bf6a-4e7a-ad45-8450617e202a None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:46.337872 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cba3594-bf6a-4e7a-ad45-8450617e202a None admin] RBAC: Adding query filter params (name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:46.338180 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cba3594-bf6a-4e7a-ad45-8450617e202a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:46.340157 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cba3594-bf6a-4e7a-ad45-8450617e202a None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:46.348754 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 296/585] 10.222.0.26 () {56 vars in 1103 bytes} [Sat Aug 6 01:20:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:46.349588 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cba3594-bf6a-4e7a-ad45-8450617e202a None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:46.371306 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84131c3-3ace-4a6d-bf2c-994a9785f15e None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:46.399458 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:46.413827 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6ffcd9-9749-4282-b5c9-2e9b0bc1d3c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:46.414402 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6ffcd9-9749-4282-b5c9-2e9b0bc1d3c2 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:46.414690 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6ffcd9-9749-4282-b5c9-2e9b0bc1d3c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:46.414964 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6ffcd9-9749-4282-b5c9-2e9b0bc1d3c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:46.425697 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6ffcd9-9749-4282-b5c9-2e9b0bc1d3c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:46.427151 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6ffcd9-9749-4282-b5c9-2e9b0bc1d3c2 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:46.441721 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-fe6ffcd9-9749-4282-b5c9-2e9b0bc1d3c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:20:46.441721 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:46.441721 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:46.441721 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:46.441721 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:46.441721 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:46.441721 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:46.441721 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:46.441721 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:46.441721 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:46.441721 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:46.441721 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:46.441721 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:20:46.441721 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:20:46.441721 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:20:46.442822 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:20:46.442822 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:46.442822 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:46.442822 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:46.442822 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:46.442822 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:46.442822 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:46.442822 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:46.442822 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:46.442822 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:46.442822 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:46.442822 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:46.442822 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:46.442822 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:46.442822 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:46.442822 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:46.442822 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:46.443802 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:20:46.443802 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:20:46.443802 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:20:46.443802 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:20:46.443802 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 290/586] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:20:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:20:46.444236 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6ffcd9-9749-4282-b5c9-2e9b0bc1d3c2 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:46.462902 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:46.471451 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cba3594-bf6a-4e7a-ad45-8450617e202a None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:46.506543 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001610cb-2978-4a92-a1c9-1103e996701d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:46.507293 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001610cb-2978-4a92-a1c9-1103e996701d None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:46.507573 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001610cb-2978-4a92-a1c9-1103e996701d None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:46.507920 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001610cb-2978-4a92-a1c9-1103e996701d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:46.516562 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001610cb-2978-4a92-a1c9-1103e996701d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:46.518034 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001610cb-2978-4a92-a1c9-1103e996701d None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:46.530631 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-001610cb-2978-4a92-a1c9-1103e996701d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 06 01:20:46.530631 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:46.530631 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:46.530631 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:46.530631 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:46.530631 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:46.530631 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:46.530631 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:46.530631 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:46.530631 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:46.530631 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:46.530631 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:46.530631 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 01:20:46.530631 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 01:20:46.530631 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 01:20:46.530631 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 01:20:46.531759 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:46.531759 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:46.531759 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 01:20:46.531759 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:20:46.531759 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 01:20:46.531759 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:20:46.531759 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:46.531759 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:46.531759 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:46.531759 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:46.531759 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:46.531759 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:46.531759 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:46.531759 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:46.531759 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:46.531759 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:46.531759 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:46.531759 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:46.532716 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:46.532716 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:46.532716 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 01:20:46.532716 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 01:20:46.532716 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 01:20:46.532716 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 01:20:46.532716 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 01:20:46.532716 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 01:20:46.532716 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 06 01:20:46.532716 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:20:46.532716 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 297/587] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Aug 6 01:20:46 2022] GET /identity/v3/users/cinder => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:20:46.533341 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001610cb-2978-4a92-a1c9-1103e996701d None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:46.553740 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6ffcd9-9749-4282-b5c9-2e9b0bc1d3c2 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:46.590411 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70525062-2cd8-42a4-91bf-c9dd6b036066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:46.591085 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70525062-2cd8-42a4-91bf-c9dd6b036066 None admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:46.591369 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70525062-2cd8-42a4-91bf-c9dd6b036066 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:46.591648 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70525062-2cd8-42a4-91bf-c9dd6b036066 None admin] PATH_INFO: `/v3/projects/d9d68d310bf24449a5fe20d2dc264a58/users/abb76a5c8ef74bf2b18c56074daaad58/roles/c9ce798acd4b482f8f391c6af97eb7a4` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:46.606370 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70525062-2cd8-42a4-91bf-c9dd6b036066 None admin] RBAC: Authorizing `identity:create_grant(project_id=d9d68d310bf24449a5fe20d2dc264a58, user_id=abb76a5c8ef74bf2b18c56074daaad58, role_id=c9ce798acd4b482f8f391c6af97eb7a4)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:46.607872 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70525062-2cd8-42a4-91bf-c9dd6b036066 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:46.620732 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 291/588] 10.222.0.26 () {58 vars in 1439 bytes} [Sat Aug 6 01:20:46 2022] PUT /identity/v3/projects/d9d68d310bf24449a5fe20d2dc264a58/users/abb76a5c8ef74bf2b18c56074daaad58/roles/c9ce798acd4b482f8f391c6af97eb7a4 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:20:46.621788 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70525062-2cd8-42a4-91bf-c9dd6b036066 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:46.653540 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001610cb-2978-4a92-a1c9-1103e996701d None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:46.668245 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:46.702566 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb25abb7-2cbd-414e-b9e9-30b206faf0a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:46.703957 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb25abb7-2cbd-414e-b9e9-30b206faf0a2 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:46.704676 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb25abb7-2cbd-414e-b9e9-30b206faf0a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:46.705207 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb25abb7-2cbd-414e-b9e9-30b206faf0a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:46.721129 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb25abb7-2cbd-414e-b9e9-30b206faf0a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:46.723584 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb25abb7-2cbd-414e-b9e9-30b206faf0a2 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:46.746765 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-fb25abb7-2cbd-414e-b9e9-30b206faf0a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:20:46.746765 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:46.746765 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:46.746765 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:46.746765 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:46.746765 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:46.746765 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:46.746765 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:46.746765 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:46.746765 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:46.746765 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:46.746765 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:46.746765 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:20:46.746765 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:20:46.746765 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:20:46.747731 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:20:46.747731 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:46.747731 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:46.747731 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:46.747731 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:46.747731 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:46.747731 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:46.747731 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:46.747731 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:46.747731 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:46.747731 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:46.747731 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:46.747731 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:46.747731 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:46.747731 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:46.747731 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:46.747731 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:46.748694 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:20:46.748694 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:20:46.748694 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:20:46.748694 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:20:46.755673 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 298/589] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:20:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:20:46.755673 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb25abb7-2cbd-414e-b9e9-30b206faf0a2 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:46.759995 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70525062-2cd8-42a4-91bf-c9dd6b036066 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:46.769375 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:46.810735 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bd1976-3de6-4232-9401-e3d688f7ad72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:46.813804 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bd1976-3de6-4232-9401-e3d688f7ad72 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:46.814094 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bd1976-3de6-4232-9401-e3d688f7ad72 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:46.814379 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bd1976-3de6-4232-9401-e3d688f7ad72 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:46.815096 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bd1976-3de6-4232-9401-e3d688f7ad72 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:46.815973 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bd1976-3de6-4232-9401-e3d688f7ad72 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:46.817916 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bd1976-3de6-4232-9401-e3d688f7ad72 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:46.832989 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 292/590] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Aug 6 01:20:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:46.833717 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bd1976-3de6-4232-9401-e3d688f7ad72 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:46.895833 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb25abb7-2cbd-414e-b9e9-30b206faf0a2 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:46.938673 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:46.942980 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cde8a04-87bd-4339-8b58-47e4a1cb994a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:46.943776 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cde8a04-87bd-4339-8b58-47e4a1cb994a None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:46.944170 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cde8a04-87bd-4339-8b58-47e4a1cb994a None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:46.944542 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cde8a04-87bd-4339-8b58-47e4a1cb994a None admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:46.945869 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cde8a04-87bd-4339-8b58-47e4a1cb994a None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:46.946239 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cde8a04-87bd-4339-8b58-47e4a1cb994a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:46.947758 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cde8a04-87bd-4339-8b58-47e4a1cb994a None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:46.961616 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bd1976-3de6-4232-9401-e3d688f7ad72 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:46.968524 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 299/591] 10.222.0.26 () {56 vars in 1137 bytes} [Sat Aug 6 01:20:46 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:46.973101 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cde8a04-87bd-4339-8b58-47e4a1cb994a None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:47.006101 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357965a2-624c-43fa-b7b8-be09da71d8b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:47.006730 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357965a2-624c-43fa-b7b8-be09da71d8b3 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:47.007091 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357965a2-624c-43fa-b7b8-be09da71d8b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:47.007340 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357965a2-624c-43fa-b7b8-be09da71d8b3 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:47.008522 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357965a2-624c-43fa-b7b8-be09da71d8b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:47.008634 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357965a2-624c-43fa-b7b8-be09da71d8b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:47.010043 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357965a2-624c-43fa-b7b8-be09da71d8b3 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:47.024016 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 293/592] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Aug 6 01:20:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:47.024860 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357965a2-624c-43fa-b7b8-be09da71d8b3 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:47.056170 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:47.106166 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cde8a04-87bd-4339-8b58-47e4a1cb994a None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:47.140646 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357965a2-624c-43fa-b7b8-be09da71d8b3 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:47.142962 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1bc296-0085-4f93-b1a9-6d17189f95db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:47.149084 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1bc296-0085-4f93-b1a9-6d17189f95db None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:47.149084 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1bc296-0085-4f93-b1a9-6d17189f95db None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:47.149084 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1bc296-0085-4f93-b1a9-6d17189f95db None admin] PATH_INFO: `/v3/projects/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:47.159664 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1bc296-0085-4f93-b1a9-6d17189f95db None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:47.161319 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1bc296-0085-4f93-b1a9-6d17189f95db None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:47.174803 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-ba1bc296-0085-4f93-b1a9-6d17189f95db None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:20:47.174803 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:47.174803 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:47.174803 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:47.174803 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:47.174803 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:47.174803 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:47.174803 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:47.174803 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:47.174803 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:47.174803 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:47.174803 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:47.174803 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 01:20:47.174803 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 01:20:47.174803 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 01:20:47.175720 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 01:20:47.175720 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:47.175720 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:47.175720 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:47.175720 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:47.175720 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:47.175720 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:47.175720 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:47.175720 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:47.175720 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:47.175720 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:47.175720 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:47.175720 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:47.175720 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:47.175720 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:47.175720 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:47.175720 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:47.176617 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 01:20:47.176617 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 01:20:47.176617 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 01:20:47.176617 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 01:20:47.176617 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 01:20:47.176617 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 01:20:47.176617 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:20:47.176617 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:20:47.176617 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 300/593] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Aug 6 01:20:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:20:47.177149 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1bc296-0085-4f93-b1a9-6d17189f95db None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:47.184392 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf640a17-692a-4682-9486-f929f413db86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:47.185064 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf640a17-692a-4682-9486-f929f413db86 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:47.185361 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf640a17-692a-4682-9486-f929f413db86 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:47.185639 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf640a17-692a-4682-9486-f929f413db86 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:47.197107 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf640a17-692a-4682-9486-f929f413db86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:47.198423 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf640a17-692a-4682-9486-f929f413db86 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:47.205814 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:47.217008 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-cf640a17-692a-4682-9486-f929f413db86 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:20:47.217008 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:47.217008 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:47.217008 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:47.217008 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:47.217008 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:47.217008 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:47.217008 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:47.217008 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:47.217008 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:47.217008 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:47.217008 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:47.217008 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:20:47.217008 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:20:47.217008 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:20:47.217997 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:20:47.217997 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:47.217997 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:47.217997 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:47.217997 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:47.217997 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:47.217997 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:47.217997 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:47.217997 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:47.217997 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:47.217997 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:47.217997 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:47.217997 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:47.217997 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:47.217997 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:47.217997 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:47.217997 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:47.218896 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:20:47.218896 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:20:47.218896 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:20:47.218896 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:20:47.218896 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 294/594] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:20:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:20:47.219579 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf640a17-692a-4682-9486-f929f413db86 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:47.313362 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1bc296-0085-4f93-b1a9-6d17189f95db None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:47.331917 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf640a17-692a-4682-9486-f929f413db86 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:47.338736 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:47.350366 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3ed62d-227f-4fad-a5ae-7499457e3cee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:47.351540 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3ed62d-227f-4fad-a5ae-7499457e3cee None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:47.352028 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3ed62d-227f-4fad-a5ae-7499457e3cee None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:47.352479 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3ed62d-227f-4fad-a5ae-7499457e3cee None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:47.365041 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3ed62d-227f-4fad-a5ae-7499457e3cee None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:47.367072 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3ed62d-227f-4fad-a5ae-7499457e3cee None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:47.370199 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c846a8-efcf-47b8-b459-2ade99c683be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:47.371110 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c846a8-efcf-47b8-b459-2ade99c683be None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:47.371397 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c846a8-efcf-47b8-b459-2ade99c683be None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:47.371677 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c846a8-efcf-47b8-b459-2ade99c683be None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:47.372423 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c846a8-efcf-47b8-b459-2ade99c683be None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:47.372733 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c846a8-efcf-47b8-b459-2ade99c683be None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:47.374179 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c846a8-efcf-47b8-b459-2ade99c683be None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:47.380559 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-ec3ed62d-227f-4fad-a5ae-7499457e3cee None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 06 01:20:47.380559 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:47.380559 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:47.380559 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:47.380559 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:47.380559 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:47.380559 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:47.380559 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:47.380559 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:47.380559 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:47.380559 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:47.380559 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:47.380559 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 01:20:47.380559 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 01:20:47.380559 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 01:20:47.380559 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 01:20:47.381552 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:47.381552 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:47.381552 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 01:20:47.381552 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:20:47.381552 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 01:20:47.381552 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:20:47.381552 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:47.381552 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:47.381552 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:47.381552 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:47.381552 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:47.381552 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:47.381552 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:47.381552 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:47.381552 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:47.381552 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:47.381552 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:47.381552 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:47.382699 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:47.382699 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:47.382699 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 01:20:47.382699 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 01:20:47.382699 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 01:20:47.382699 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 01:20:47.382699 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 01:20:47.382699 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 01:20:47.382699 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 06 01:20:47.382699 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:20:47.384599 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 301/595] 10.222.0.26 () {56 vars in 1102 bytes} [Sat Aug 6 01:20:47 2022] GET /identity/v3/users/neutron => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:20:47.385539 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3ed62d-227f-4fad-a5ae-7499457e3cee None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:47.393887 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 295/596] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Aug 6 01:20:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:47.393887 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7ea451-d0f7-4e24-91e1-7413c53cbe43 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:47.393887 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7ea451-d0f7-4e24-91e1-7413c53cbe43 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:47.393887 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7ea451-d0f7-4e24-91e1-7413c53cbe43 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:47.397200 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 296/597] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:20:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:20:47.398126 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7ea451-d0f7-4e24-91e1-7413c53cbe43 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:47.466148 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:47.504665 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3ed62d-227f-4fad-a5ae-7499457e3cee None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:47.512059 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7ea451-d0f7-4e24-91e1-7413c53cbe43 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:47.540455 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8d1344-1ab2-4e14-b8d6-7ad09a1ff7c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:47.541095 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8d1344-1ab2-4e14-b8d6-7ad09a1ff7c1 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:47.541385 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8d1344-1ab2-4e14-b8d6-7ad09a1ff7c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:47.541670 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8d1344-1ab2-4e14-b8d6-7ad09a1ff7c1 None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:47.542386 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8d1344-1ab2-4e14-b8d6-7ad09a1ff7c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:47.542724 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8d1344-1ab2-4e14-b8d6-7ad09a1ff7c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:47.544062 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8d1344-1ab2-4e14-b8d6-7ad09a1ff7c1 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:47.548598 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e811cb-cb04-4ade-999e-05e71e70cd99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:47.549248 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e811cb-cb04-4ade-999e-05e71e70cd99 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:47.549533 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e811cb-cb04-4ade-999e-05e71e70cd99 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:47.549809 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e811cb-cb04-4ade-999e-05e71e70cd99 None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:47.550590 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e811cb-cb04-4ade-999e-05e71e70cd99 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:47.550894 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e811cb-cb04-4ade-999e-05e71e70cd99 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:47.552297 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e811cb-cb04-4ade-999e-05e71e70cd99 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:47.561078 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 302/598] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Aug 6 01:20:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:47.561078 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f5b680-13b6-458f-a4a5-6790558857be None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:47.561078 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f5b680-13b6-458f-a4a5-6790558857be None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:47.561078 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f5b680-13b6-458f-a4a5-6790558857be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:47.578964 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 297/599] 10.222.0.26 () {56 vars in 1139 bytes} [Sat Aug 6 01:20:47 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:47.578964 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e811cb-cb04-4ade-999e-05e71e70cd99 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:47.580249 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-60f5b680-13b6-458f-a4a5-6790558857be None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:20:47.581488 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-60f5b680-13b6-458f-a4a5-6790558857be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:20:47.624425 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:47.653102 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 303/600] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:20:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:20:47.653502 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f5b680-13b6-458f-a4a5-6790558857be None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:47.696690 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e811cb-cb04-4ade-999e-05e71e70cd99 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:47.713764 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:47.732004 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20c2564-b30d-481f-adf0-9f04ad72b85d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:47.732600 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20c2564-b30d-481f-adf0-9f04ad72b85d None admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:47.732885 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20c2564-b30d-481f-adf0-9f04ad72b85d None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:47.733194 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20c2564-b30d-481f-adf0-9f04ad72b85d None admin] PATH_INFO: `/v3/projects/d9d68d310bf24449a5fe20d2dc264a58/users/b87adc1d5bc240e99f95f6595b350be4/roles/c9ce798acd4b482f8f391c6af97eb7a4` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:47.748202 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20c2564-b30d-481f-adf0-9f04ad72b85d None admin] RBAC: Authorizing `identity:create_grant(project_id=d9d68d310bf24449a5fe20d2dc264a58, user_id=b87adc1d5bc240e99f95f6595b350be4, role_id=c9ce798acd4b482f8f391c6af97eb7a4)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:47.749674 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20c2564-b30d-481f-adf0-9f04ad72b85d None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:47.760636 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f5b680-13b6-458f-a4a5-6790558857be None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:47.764378 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 298/601] 10.222.0.26 () {58 vars in 1439 bytes} [Sat Aug 6 01:20:47 2022] PUT /identity/v3/projects/d9d68d310bf24449a5fe20d2dc264a58/users/b87adc1d5bc240e99f95f6595b350be4/roles/c9ce798acd4b482f8f391c6af97eb7a4 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:20:47.765354 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20c2564-b30d-481f-adf0-9f04ad72b85d None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:47.809251 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d09f0f-8b2c-4f10-a447-84cbd8f219cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:47.810059 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d09f0f-8b2c-4f10-a447-84cbd8f219cc None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:47.810416 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d09f0f-8b2c-4f10-a447-84cbd8f219cc None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:47.810808 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d09f0f-8b2c-4f10-a447-84cbd8f219cc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:47.819215 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d09f0f-8b2c-4f10-a447-84cbd8f219cc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:47.820775 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d09f0f-8b2c-4f10-a447-84cbd8f219cc None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:47.836048 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-75d09f0f-8b2c-4f10-a447-84cbd8f219cc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:20:47.836048 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:47.836048 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:47.836048 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:47.836048 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:47.836048 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:47.836048 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:47.836048 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:47.836048 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:47.836048 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:47.836048 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:47.836048 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:47.836048 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 01:20:47.836048 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 01:20:47.836048 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 01:20:47.837022 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 01:20:47.837022 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:47.837022 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:47.837022 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:47.837022 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:47.837022 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:47.837022 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:47.837022 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:47.837022 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:47.837022 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:47.837022 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:47.837022 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:47.837022 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:47.837022 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:47.837022 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:47.837022 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:47.837022 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:47.837959 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 01:20:47.837959 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 01:20:47.837959 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 01:20:47.837959 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 01:20:47.837959 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 01:20:47.837959 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 01:20:47.837959 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:20:47.837959 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:20:47.841259 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 304/602] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Aug 6 01:20:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:20:47.842519 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d09f0f-8b2c-4f10-a447-84cbd8f219cc None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:47.896080 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:47.900991 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20c2564-b30d-481f-adf0-9f04ad72b85d None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:47.942692 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe181c2-57d8-47d0-96b8-085761e16c62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:47.943413 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe181c2-57d8-47d0-96b8-085761e16c62 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:47.943813 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe181c2-57d8-47d0-96b8-085761e16c62 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:47.944163 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe181c2-57d8-47d0-96b8-085761e16c62 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:47.958668 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe181c2-57d8-47d0-96b8-085761e16c62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:47.960245 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe181c2-57d8-47d0-96b8-085761e16c62 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:47.978462 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-5fe181c2-57d8-47d0-96b8-085761e16c62 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:20:47.978462 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:47.978462 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:47.978462 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:47.978462 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:47.978462 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:47.978462 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:47.978462 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:47.978462 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:47.978462 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:47.978462 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:47.978462 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:47.978462 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:20:47.978462 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:20:47.978462 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:20:47.979410 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:20:47.979410 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:47.979410 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:47.979410 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:47.979410 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:47.979410 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:47.979410 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:47.979410 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:47.979410 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:47.979410 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:47.979410 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:47.979410 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:47.979410 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:47.979410 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:47.979410 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:47.979410 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:47.979410 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:47.980955 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:20:47.980955 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:20:47.980955 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:20:47.980955 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:20:47.980955 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 299/603] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:20:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:20:47.984798 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe181c2-57d8-47d0-96b8-085761e16c62 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:47.999529 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d09f0f-8b2c-4f10-a447-84cbd8f219cc None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:48.005357 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:48.047665 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e313f14-7e53-4031-ac4f-b94bf2a958f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:48.048312 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e313f14-7e53-4031-ac4f-b94bf2a958f8 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:48.048850 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e313f14-7e53-4031-ac4f-b94bf2a958f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:48.049160 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e313f14-7e53-4031-ac4f-b94bf2a958f8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:48.064290 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e313f14-7e53-4031-ac4f-b94bf2a958f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:20:48.073298 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-5e313f14-7e53-4031-ac4f-b94bf2a958f8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 06 01:20:48.073298 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:48.073298 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:48.073298 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:48.073298 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:48.073298 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:48.073298 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:48.073298 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:48.073298 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:48.073298 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:48.073298 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:48.073298 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:48.073298 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:20:48.073298 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:20:48.073298 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:20:48.073298 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:20:48.074279 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:20:48.074279 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:20:48.074279 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:20:48.074279 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:20:48.074279 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:48.074279 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:48.074279 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:48.074279 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:48.074279 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:48.074279 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:48.074279 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:48.074279 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:48.074279 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:48.074279 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:48.074279 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:48.074279 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:48.075158 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:48.075158 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:48.075158 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:48.075158 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:48.075158 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:20:48.075158 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:20:48.075158 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:20:48.075158 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:20:48.075158 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:20:48.075158 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:20:48.075158 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 06 01:20:48.075158 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:20:48.078449 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 305/604] 10.222.0.26 () {56 vars in 1102 bytes} [Sat Aug 6 01:20:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:20:48.079678 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e313f14-7e53-4031-ac4f-b94bf2a958f8 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:48.102000 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe181c2-57d8-47d0-96b8-085761e16c62 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:48.140772 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263c4126-87d8-4062-84cf-643d2443bdd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:48.141757 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263c4126-87d8-4062-84cf-643d2443bdd0 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:48.142149 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263c4126-87d8-4062-84cf-643d2443bdd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:48.142529 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263c4126-87d8-4062-84cf-643d2443bdd0 None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:48.143530 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263c4126-87d8-4062-84cf-643d2443bdd0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:48.143943 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263c4126-87d8-4062-84cf-643d2443bdd0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:48.145627 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263c4126-87d8-4062-84cf-643d2443bdd0 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:48.160669 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 300/605] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Aug 6 01:20:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:48.161791 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263c4126-87d8-4062-84cf-643d2443bdd0 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:48.207298 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:48.217498 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e313f14-7e53-4031-ac4f-b94bf2a958f8 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:48.255022 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ab134c-4a98-4d6f-bf22-802b3fd9e124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:48.255717 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ab134c-4a98-4d6f-bf22-802b3fd9e124 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:48.256103 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ab134c-4a98-4d6f-bf22-802b3fd9e124 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:48.256480 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ab134c-4a98-4d6f-bf22-802b3fd9e124 None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:48.257353 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ab134c-4a98-4d6f-bf22-802b3fd9e124 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:48.257816 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ab134c-4a98-4d6f-bf22-802b3fd9e124 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:48.259318 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ab134c-4a98-4d6f-bf22-802b3fd9e124 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:48.263279 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:48.276093 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 306/606] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Aug 6 01:20:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:48.277495 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ab134c-4a98-4d6f-bf22-802b3fd9e124 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:48.288469 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263c4126-87d8-4062-84cf-643d2443bdd0 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:48.335834 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31ee928-067f-44a7-be1f-c4f2681015bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:48.336466 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31ee928-067f-44a7-be1f-c4f2681015bc None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:48.336754 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31ee928-067f-44a7-be1f-c4f2681015bc None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:48.337033 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31ee928-067f-44a7-be1f-c4f2681015bc None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:48.337776 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31ee928-067f-44a7-be1f-c4f2681015bc None admin] RBAC: Adding query filter params (name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:48.338085 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31ee928-067f-44a7-be1f-c4f2681015bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:48.340108 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31ee928-067f-44a7-be1f-c4f2681015bc None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:48.351382 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 301/607] 10.222.0.26 () {56 vars in 1103 bytes} [Sat Aug 6 01:20:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:48.352259 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31ee928-067f-44a7-be1f-c4f2681015bc None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:48.392622 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ab134c-4a98-4d6f-bf22-802b3fd9e124 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:48.432617 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b3d888-8ce7-4b07-98d8-9b075da5f549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:48.434403 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b3d888-8ce7-4b07-98d8-9b075da5f549 None admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:48.434691 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b3d888-8ce7-4b07-98d8-9b075da5f549 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:48.434970 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b3d888-8ce7-4b07-98d8-9b075da5f549 None admin] PATH_INFO: `/v3/projects/d9d68d310bf24449a5fe20d2dc264a58/users/1be1e6173e374a60b9e1fbf51a1ab571/roles/c9ce798acd4b482f8f391c6af97eb7a4` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:48.457296 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b3d888-8ce7-4b07-98d8-9b075da5f549 None admin] RBAC: Authorizing `identity:create_grant(project_id=d9d68d310bf24449a5fe20d2dc264a58, user_id=1be1e6173e374a60b9e1fbf51a1ab571, role_id=c9ce798acd4b482f8f391c6af97eb7a4)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:48.458841 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b3d888-8ce7-4b07-98d8-9b075da5f549 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:48.474983 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 307/608] 10.222.0.26 () {58 vars in 1439 bytes} [Sat Aug 6 01:20:48 2022] PUT /identity/v3/projects/d9d68d310bf24449a5fe20d2dc264a58/users/1be1e6173e374a60b9e1fbf51a1ab571/roles/c9ce798acd4b482f8f391c6af97eb7a4 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:20:48.476607 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b3d888-8ce7-4b07-98d8-9b075da5f549 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:48.481611 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:48.486404 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31ee928-067f-44a7-be1f-c4f2681015bc None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:48.531209 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fc4b29-230d-40aa-9b17-b7db195557b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:48.532337 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fc4b29-230d-40aa-9b17-b7db195557b5 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:48.532828 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fc4b29-230d-40aa-9b17-b7db195557b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:48.533338 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fc4b29-230d-40aa-9b17-b7db195557b5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:48.541360 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fc4b29-230d-40aa-9b17-b7db195557b5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:48.543360 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fc4b29-230d-40aa-9b17-b7db195557b5 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:48.546764 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:48.560499 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-86fc4b29-230d-40aa-9b17-b7db195557b5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:20:48.560499 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:48.560499 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:48.560499 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:48.560499 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:48.560499 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:48.560499 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:48.560499 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:48.560499 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:48.560499 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:48.560499 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:48.560499 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:48.560499 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 01:20:48.560499 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 01:20:48.560499 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 01:20:48.561728 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 01:20:48.561728 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:48.561728 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:48.561728 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:48.561728 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:48.561728 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:48.561728 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:48.561728 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:48.561728 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:48.561728 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:48.561728 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:48.561728 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:48.561728 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:48.561728 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:48.561728 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:48.561728 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:48.561728 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:48.562816 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 01:20:48.562816 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 01:20:48.562816 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 01:20:48.562816 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 01:20:48.562816 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 01:20:48.562816 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 01:20:48.562816 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:20:48.562816 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:20:48.564526 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 302/609] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Aug 6 01:20:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:20:48.570930 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fc4b29-230d-40aa-9b17-b7db195557b5 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:48.634177 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b3d888-8ce7-4b07-98d8-9b075da5f549 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:48.706756 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460574a0-41d1-4b33-a4b4-6408caf83d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:48.707424 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460574a0-41d1-4b33-a4b4-6408caf83d2d None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:48.707891 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460574a0-41d1-4b33-a4b4-6408caf83d2d None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:48.708176 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460574a0-41d1-4b33-a4b4-6408caf83d2d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:48.720599 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460574a0-41d1-4b33-a4b4-6408caf83d2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:48.722180 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460574a0-41d1-4b33-a4b4-6408caf83d2d None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:48.738147 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-460574a0-41d1-4b33-a4b4-6408caf83d2d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:20:48.738147 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:48.738147 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:48.738147 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:48.738147 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:48.738147 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:48.738147 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:48.738147 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:48.738147 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:48.738147 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:48.738147 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:48.738147 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:48.738147 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:20:48.738147 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:20:48.738147 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:20:48.739086 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:20:48.739086 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:48.739086 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:48.739086 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:48.739086 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:48.739086 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:48.739086 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:48.739086 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:48.739086 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:48.739086 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:48.739086 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:48.739086 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:48.739086 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:48.739086 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:48.739086 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:48.739086 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:48.739086 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:48.739962 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:20:48.739962 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:20:48.739962 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:20:48.739962 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:20:48.739962 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 308/610] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:20:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:20:48.744824 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460574a0-41d1-4b33-a4b4-6408caf83d2d None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:48.752734 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fc4b29-230d-40aa-9b17-b7db195557b5 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:48.792362 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a0d562-be8b-40b4-82a3-9640eedfdec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:48.793491 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a0d562-be8b-40b4-82a3-9640eedfdec8 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:48.793958 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a0d562-be8b-40b4-82a3-9640eedfdec8 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:48.794410 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a0d562-be8b-40b4-82a3-9640eedfdec8 None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:48.795339 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a0d562-be8b-40b4-82a3-9640eedfdec8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:48.795851 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a0d562-be8b-40b4-82a3-9640eedfdec8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:48.797571 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a0d562-be8b-40b4-82a3-9640eedfdec8 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:48.811407 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 303/611] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Aug 6 01:20:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:48.812914 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31a6541-1c9f-4137-b498-b1965a0259b0 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:48.813434 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31a6541-1c9f-4137-b498-b1965a0259b0 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:48.813967 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31a6541-1c9f-4137-b498-b1965a0259b0 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:48.815389 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 304/612] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:20:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:20:48.817036 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31a6541-1c9f-4137-b498-b1965a0259b0 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:48.818712 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:48.875380 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460574a0-41d1-4b33-a4b4-6408caf83d2d None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:48.883723 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:48.913288 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c357d2e7-ab98-4167-9ec3-f41b67226310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:48.914563 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c357d2e7-ab98-4167-9ec3-f41b67226310 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:48.915277 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c357d2e7-ab98-4167-9ec3-f41b67226310 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:48.916231 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c357d2e7-ab98-4167-9ec3-f41b67226310 None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:48.917453 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c357d2e7-ab98-4167-9ec3-f41b67226310 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:48.917972 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c357d2e7-ab98-4167-9ec3-f41b67226310 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:48.920396 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c357d2e7-ab98-4167-9ec3-f41b67226310 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:48.929467 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31a6541-1c9f-4137-b498-b1965a0259b0 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:48.937437 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 309/613] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Aug 6 01:20:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:20:48.938902 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d45a44-7321-4632-8ca6-782405a6b4fa None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:48.939812 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d45a44-7321-4632-8ca6-782405a6b4fa None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:48.940228 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d45a44-7321-4632-8ca6-782405a6b4fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:48.963213 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-95d45a44-7321-4632-8ca6-782405a6b4fa None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:20:48.965628 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-95d45a44-7321-4632-8ca6-782405a6b4fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:20:48.972947 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dc51c0-b31b-492f-9504-45af5e15cdc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:48.973846 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dc51c0-b31b-492f-9504-45af5e15cdc5 None admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:48.974292 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dc51c0-b31b-492f-9504-45af5e15cdc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:48.974705 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dc51c0-b31b-492f-9504-45af5e15cdc5 None admin] PATH_INFO: `/v3/projects/d9d68d310bf24449a5fe20d2dc264a58/users/4ac0e31c84b4445fa167bbe4954066c2/roles/c9ce798acd4b482f8f391c6af97eb7a4` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:48.990919 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dc51c0-b31b-492f-9504-45af5e15cdc5 None admin] RBAC: Authorizing `identity:create_grant(project_id=d9d68d310bf24449a5fe20d2dc264a58, user_id=4ac0e31c84b4445fa167bbe4954066c2, role_id=c9ce798acd4b482f8f391c6af97eb7a4)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:48.992599 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dc51c0-b31b-492f-9504-45af5e15cdc5 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:49.009600 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 305/614] 10.222.0.26 () {58 vars in 1439 bytes} [Sat Aug 6 01:20:48 2022] PUT /identity/v3/projects/d9d68d310bf24449a5fe20d2dc264a58/users/4ac0e31c84b4445fa167bbe4954066c2/roles/c9ce798acd4b482f8f391c6af97eb7a4 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:20:49.011010 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dc51c0-b31b-492f-9504-45af5e15cdc5 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:49.055869 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 310/615] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:20:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:20:49.114688 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d45a44-7321-4632-8ca6-782405a6b4fa None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:49.177701 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dc51c0-b31b-492f-9504-45af5e15cdc5 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:49.208085 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:49.219110 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:49.243271 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8400ddd6-76d5-4b20-87b3-8f6f3fc043f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:49.243923 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8400ddd6-76d5-4b20-87b3-8f6f3fc043f1 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:49.244206 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8400ddd6-76d5-4b20-87b3-8f6f3fc043f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:49.244487 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8400ddd6-76d5-4b20-87b3-8f6f3fc043f1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:49.253567 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8400ddd6-76d5-4b20-87b3-8f6f3fc043f1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:49.255074 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8400ddd6-76d5-4b20-87b3-8f6f3fc043f1 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:49.272106 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-8400ddd6-76d5-4b20-87b3-8f6f3fc043f1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 06 01:20:49.272106 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:49.272106 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:49.272106 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:49.272106 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:49.272106 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:49.272106 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:49.272106 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:49.272106 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:49.272106 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:49.272106 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:49.272106 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:49.272106 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 01:20:49.272106 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 01:20:49.272106 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 01:20:49.272106 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 01:20:49.273123 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:49.273123 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:49.273123 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 01:20:49.273123 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:20:49.273123 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 01:20:49.273123 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:20:49.273123 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:49.273123 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:49.273123 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:49.273123 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:49.273123 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:49.273123 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:49.273123 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:49.273123 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:49.273123 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:49.273123 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:49.273123 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:49.273123 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:49.274071 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:49.274071 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:49.274071 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 01:20:49.274071 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 01:20:49.274071 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 01:20:49.274071 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 01:20:49.274071 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 01:20:49.274071 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 01:20:49.274071 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 06 01:20:49.274071 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:20:49.274071 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 306/616] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Aug 6 01:20:49 2022] GET /identity/v3/users/swift => generated 83 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:20:49.280489 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d45a44-7321-4632-8ca6-782405a6b4fa None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:49.282118 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8400ddd6-76d5-4b20-87b3-8f6f3fc043f1 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:49.335871 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa35b2b-a66d-4935-9fe2-2d0467483489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:49.336450 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa35b2b-a66d-4935-9fe2-2d0467483489 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:49.336734 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa35b2b-a66d-4935-9fe2-2d0467483489 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:49.337009 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa35b2b-a66d-4935-9fe2-2d0467483489 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:49.359693 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa35b2b-a66d-4935-9fe2-2d0467483489 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:20:49.367934 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-5aa35b2b-a66d-4935-9fe2-2d0467483489 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 06 01:20:49.367934 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:49.367934 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:49.367934 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:49.367934 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:49.367934 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:49.367934 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:49.367934 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:49.367934 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:49.367934 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:49.367934 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:49.367934 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:49.367934 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:20:49.367934 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:20:49.367934 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:20:49.367934 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:20:49.368910 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:20:49.368910 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:20:49.368910 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:20:49.368910 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:20:49.368910 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:49.368910 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:49.368910 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:49.368910 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:49.368910 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:49.368910 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:49.368910 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:49.368910 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:49.368910 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:49.368910 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:49.368910 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:49.368910 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:49.369808 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:49.369808 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:49.369808 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:49.369808 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:49.369808 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:20:49.369808 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:20:49.369808 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:20:49.369808 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:20:49.369808 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:20:49.369808 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:20:49.369808 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 06 01:20:49.369808 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:20:49.378994 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 311/617] 10.222.0.26 () {56 vars in 1102 bytes} [Sat Aug 6 01:20:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:20:49.382949 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa35b2b-a66d-4935-9fe2-2d0467483489 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:49.472527 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8400ddd6-76d5-4b20-87b3-8f6f3fc043f1 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:49.537565 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0a1aec-2f71-42b6-87e0-c650c691329e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:49.538192 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0a1aec-2f71-42b6-87e0-c650c691329e None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:49.538478 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0a1aec-2f71-42b6-87e0-c650c691329e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:49.538754 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0a1aec-2f71-42b6-87e0-c650c691329e None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:49.540453 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0a1aec-2f71-42b6-87e0-c650c691329e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:49.540773 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0a1aec-2f71-42b6-87e0-c650c691329e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:49.542231 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0a1aec-2f71-42b6-87e0-c650c691329e None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:49.568088 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa35b2b-a66d-4935-9fe2-2d0467483489 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:49.572627 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 307/618] 10.222.0.26 () {56 vars in 1135 bytes} [Sat Aug 6 01:20:49 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:49.581766 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0a1aec-2f71-42b6-87e0-c650c691329e None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:49.584168 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:49.631262 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:49.638512 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6879d4d9-c3ca-45e3-9beb-f7a9bc03e8cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:49.639143 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6879d4d9-c3ca-45e3-9beb-f7a9bc03e8cc None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:49.639426 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6879d4d9-c3ca-45e3-9beb-f7a9bc03e8cc None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:49.639700 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6879d4d9-c3ca-45e3-9beb-f7a9bc03e8cc None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:49.640430 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6879d4d9-c3ca-45e3-9beb-f7a9bc03e8cc None admin] RBAC: Adding query filter params (name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:49.640737 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6879d4d9-c3ca-45e3-9beb-f7a9bc03e8cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:49.645149 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6879d4d9-c3ca-45e3-9beb-f7a9bc03e8cc None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:49.656252 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 312/619] 10.222.0.26 () {56 vars in 1103 bytes} [Sat Aug 6 01:20:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:49.664004 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6879d4d9-c3ca-45e3-9beb-f7a9bc03e8cc None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:49.770347 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0a1aec-2f71-42b6-87e0-c650c691329e None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:49.811098 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686d0db7-9c2f-4126-af66-ab25f349077b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:49.811747 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686d0db7-9c2f-4126-af66-ab25f349077b None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:49.812032 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686d0db7-9c2f-4126-af66-ab25f349077b None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:49.812314 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686d0db7-9c2f-4126-af66-ab25f349077b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:49.824028 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686d0db7-9c2f-4126-af66-ab25f349077b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:49.825756 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686d0db7-9c2f-4126-af66-ab25f349077b None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:49.844104 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-686d0db7-9c2f-4126-af66-ab25f349077b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:20:49.844104 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:49.844104 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:49.844104 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:49.844104 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:49.844104 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:49.844104 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:49.844104 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:49.844104 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:49.844104 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:49.844104 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:49.844104 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:49.844104 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:20:49.844104 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:20:49.844104 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:20:49.845227 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:20:49.845227 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:49.845227 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:49.845227 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:49.845227 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:49.845227 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:49.845227 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:49.845227 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:49.845227 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:49.845227 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:49.845227 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:49.845227 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:49.845227 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:49.845227 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:49.845227 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:49.845227 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:49.845227 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:49.846133 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:20:49.846133 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:20:49.846133 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:20:49.846133 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:20:49.846133 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 308/620] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:20:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:20:49.852123 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686d0db7-9c2f-4126-af66-ab25f349077b None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:49.891621 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6879d4d9-c3ca-45e3-9beb-f7a9bc03e8cc None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:49.955099 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0ed640-91c1-4cd8-8679-1d9c93c6881d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:49.956679 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0ed640-91c1-4cd8-8679-1d9c93c6881d None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:49.957210 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0ed640-91c1-4cd8-8679-1d9c93c6881d None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:49.957783 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0ed640-91c1-4cd8-8679-1d9c93c6881d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:49.979611 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:49.986130 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0ed640-91c1-4cd8-8679-1d9c93c6881d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:49.987840 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0ed640-91c1-4cd8-8679-1d9c93c6881d None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:49.995562 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686d0db7-9c2f-4126-af66-ab25f349077b None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:50.018553 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-0f0ed640-91c1-4cd8-8679-1d9c93c6881d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:20:50.018553 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:50.018553 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:50.018553 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:50.018553 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:50.018553 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:50.018553 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:50.018553 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:50.018553 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:50.018553 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:50.018553 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:50.018553 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:50.018553 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:20:50.018553 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:20:50.018553 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:20:50.019511 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:20:50.019511 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:50.019511 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:50.019511 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:50.019511 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:50.019511 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:50.019511 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:50.019511 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:50.019511 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:50.019511 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:50.019511 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:50.019511 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:50.019511 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:50.019511 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:50.019511 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:50.019511 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:50.019511 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:50.020397 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:20:50.020397 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:20:50.020397 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:20:50.020397 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:20:50.028344 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 313/621] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:20:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 359 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:20:50.029548 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0ed640-91c1-4cd8-8679-1d9c93c6881d None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:50.032114 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db230e0d-065c-45e9-b12b-61a9a2cd5535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:50.032857 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db230e0d-065c-45e9-b12b-61a9a2cd5535 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:50.033294 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db230e0d-065c-45e9-b12b-61a9a2cd5535 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:50.033680 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db230e0d-065c-45e9-b12b-61a9a2cd5535 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:50.034484 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db230e0d-065c-45e9-b12b-61a9a2cd5535 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:50.041097 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db230e0d-065c-45e9-b12b-61a9a2cd5535 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:50.041097 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db230e0d-065c-45e9-b12b-61a9a2cd5535 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:50.053166 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 309/622] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Aug 6 01:20:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:50.055243 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db230e0d-065c-45e9-b12b-61a9a2cd5535 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:50.080739 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:50.175437 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0ed640-91c1-4cd8-8679-1d9c93c6881d None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:50.175763 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db230e0d-065c-45e9-b12b-61a9a2cd5535 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:50.211506 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8112b71-b123-4161-89fa-93ec3c170e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:50.212125 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8112b71-b123-4161-89fa-93ec3c170e2b None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:50.212405 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8112b71-b123-4161-89fa-93ec3c170e2b None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:50.212725 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8112b71-b123-4161-89fa-93ec3c170e2b None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:50.213523 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8112b71-b123-4161-89fa-93ec3c170e2b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:50.213927 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8112b71-b123-4161-89fa-93ec3c170e2b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:50.216016 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8112b71-b123-4161-89fa-93ec3c170e2b None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:50.221144 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fed70e-4b97-462f-b253-074741529417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:50.222223 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fed70e-4b97-462f-b253-074741529417 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:50.223181 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fed70e-4b97-462f-b253-074741529417 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:50.223567 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fed70e-4b97-462f-b253-074741529417 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:50.229025 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 314/623] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Aug 6 01:20:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:50.230564 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b52be1-139f-466f-a2e9-f9efbef4338d None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:50.230902 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b52be1-139f-466f-a2e9-f9efbef4338d None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:50.231254 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b52be1-139f-466f-a2e9-f9efbef4338d None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:50.232514 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fed70e-4b97-462f-b253-074741529417 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:50.234278 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fed70e-4b97-462f-b253-074741529417 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:50.236662 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 315/624] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:20:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:20:50.237396 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:50.238524 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b52be1-139f-466f-a2e9-f9efbef4338d None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:50.249381 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-a6fed70e-4b97-462f-b253-074741529417 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:20:50.249381 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:50.249381 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:50.249381 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:50.249381 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:50.249381 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:50.249381 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:50.249381 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:50.249381 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:50.249381 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:50.249381 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:50.249381 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:50.249381 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 01:20:50.249381 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 01:20:50.249381 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 01:20:50.250294 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 01:20:50.250294 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:50.250294 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:50.250294 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:50.250294 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:50.250294 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:50.250294 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:50.250294 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:50.250294 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:50.250294 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:50.250294 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:50.250294 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:50.250294 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:50.250294 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:50.250294 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:50.250294 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:50.250294 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:50.251169 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 01:20:50.251169 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 01:20:50.251169 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 01:20:50.251169 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 01:20:50.251169 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 01:20:50.251169 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 01:20:50.251169 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:20:50.251169 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:20:50.259465 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 310/625] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Aug 6 01:20:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:20:50.259465 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07391038-26b9-42e8-98b7-9aca7072eed0 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:50.259465 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07391038-26b9-42e8-98b7-9aca7072eed0 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:50.259465 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07391038-26b9-42e8-98b7-9aca7072eed0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:50.286082 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-07391038-26b9-42e8-98b7-9aca7072eed0 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:20:50.287416 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-07391038-26b9-42e8-98b7-9aca7072eed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:20:50.347819 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:50.371939 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b52be1-139f-466f-a2e9-f9efbef4338d None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:50.381081 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 311/626] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:20:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:20:50.381081 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07391038-26b9-42e8-98b7-9aca7072eed0 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:50.442009 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0fb631-08df-4307-bb94-bb2eff586d41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:50.443151 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0fb631-08df-4307-bb94-bb2eff586d41 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:50.443640 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0fb631-08df-4307-bb94-bb2eff586d41 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:50.444120 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0fb631-08df-4307-bb94-bb2eff586d41 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:50.454621 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0fb631-08df-4307-bb94-bb2eff586d41 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:50.456835 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0fb631-08df-4307-bb94-bb2eff586d41 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:50.469655 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-9f0fb631-08df-4307-bb94-bb2eff586d41 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 06 01:20:50.469655 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:50.469655 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:50.469655 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:50.469655 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:50.469655 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:50.469655 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:50.469655 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:50.469655 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:50.469655 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:50.469655 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:50.469655 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:50.469655 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 01:20:50.469655 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 01:20:50.469655 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 01:20:50.469655 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 01:20:50.470651 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:50.470651 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:50.470651 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 01:20:50.470651 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:20:50.470651 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 01:20:50.470651 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:20:50.470651 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:50.470651 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:50.470651 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:50.470651 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:50.470651 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:50.470651 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:50.470651 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:50.470651 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:50.470651 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:50.470651 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:50.470651 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:50.470651 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:50.472205 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:50.472205 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:50.472205 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 01:20:50.472205 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 01:20:50.472205 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 01:20:50.472205 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 01:20:50.472205 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 01:20:50.472205 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 01:20:50.472205 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 06 01:20:50.472205 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:20:50.472205 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 316/627] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Aug 6 01:20:50 2022] GET /identity/v3/users/glance => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:20:50.472205 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0fb631-08df-4307-bb94-bb2eff586d41 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:50.514029 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07391038-26b9-42e8-98b7-9aca7072eed0 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:50.554513 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:50.563740 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921c0fb5-c6f5-41f8-ba7c-5fee3bad8dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:50.564875 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921c0fb5-c6f5-41f8-ba7c-5fee3bad8dc7 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:50.565394 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921c0fb5-c6f5-41f8-ba7c-5fee3bad8dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:50.566343 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921c0fb5-c6f5-41f8-ba7c-5fee3bad8dc7 None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:50.567282 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921c0fb5-c6f5-41f8-ba7c-5fee3bad8dc7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:50.567770 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921c0fb5-c6f5-41f8-ba7c-5fee3bad8dc7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:50.569914 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921c0fb5-c6f5-41f8-ba7c-5fee3bad8dc7 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:50.581897 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0fb631-08df-4307-bb94-bb2eff586d41 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:50.595779 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 312/628] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Aug 6 01:20:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:50.595779 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921c0fb5-c6f5-41f8-ba7c-5fee3bad8dc7 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:50.631514 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0345db-57a5-4de0-996a-b34ea64f9930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:50.632660 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0345db-57a5-4de0-996a-b34ea64f9930 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:50.633124 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0345db-57a5-4de0-996a-b34ea64f9930 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:50.634449 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0345db-57a5-4de0-996a-b34ea64f9930 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:50.645094 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:50.650598 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0345db-57a5-4de0-996a-b34ea64f9930 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:20:50.656907 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-fc0345db-57a5-4de0-996a-b34ea64f9930 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 06 01:20:50.656907 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:50.656907 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:50.656907 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:50.656907 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:50.656907 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:50.656907 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:50.656907 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:50.656907 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:50.656907 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:50.656907 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:50.656907 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:50.656907 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:20:50.656907 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:20:50.656907 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:20:50.656907 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:20:50.658052 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:20:50.658052 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:20:50.658052 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:20:50.658052 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:20:50.658052 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:50.658052 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:50.658052 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:50.658052 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:50.658052 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:50.658052 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:50.658052 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:50.658052 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:50.658052 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:50.658052 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:50.658052 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:50.658052 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:50.659074 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:50.659074 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:50.659074 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:50.659074 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:50.659074 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:20:50.659074 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:20:50.659074 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:20:50.659074 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:20:50.659074 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:20:50.659074 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:20:50.659074 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 06 01:20:50.659074 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:20:50.661786 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 317/629] 10.222.0.26 () {56 vars in 1102 bytes} [Sat Aug 6 01:20:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:20:50.662750 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0345db-57a5-4de0-996a-b34ea64f9930 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:50.729334 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921c0fb5-c6f5-41f8-ba7c-5fee3bad8dc7 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:50.775706 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0085aeb1-3ee0-408c-a506-28013af5c749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:50.777112 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0085aeb1-3ee0-408c-a506-28013af5c749 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:50.777416 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0345db-57a5-4de0-996a-b34ea64f9930 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:50.777848 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0085aeb1-3ee0-408c-a506-28013af5c749 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:50.778268 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0085aeb1-3ee0-408c-a506-28013af5c749 None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:50.779466 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0085aeb1-3ee0-408c-a506-28013af5c749 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:50.779924 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0085aeb1-3ee0-408c-a506-28013af5c749 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:50.782441 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0085aeb1-3ee0-408c-a506-28013af5c749 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:50.805472 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 313/630] 10.222.0.26 () {56 vars in 1137 bytes} [Sat Aug 6 01:20:50 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:50.806825 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0085aeb1-3ee0-408c-a506-28013af5c749 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:50.825503 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f074ca-c261-43d5-a592-dfa0304dce74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:50.826656 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f074ca-c261-43d5-a592-dfa0304dce74 None admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:50.827476 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f074ca-c261-43d5-a592-dfa0304dce74 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:50.827909 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f074ca-c261-43d5-a592-dfa0304dce74 None admin] PATH_INFO: `/v3/projects/d9d68d310bf24449a5fe20d2dc264a58/users/2b23f4478f49471fba6f43e6e4be99da/roles/c9ce798acd4b482f8f391c6af97eb7a4` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:50.849913 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f074ca-c261-43d5-a592-dfa0304dce74 None admin] RBAC: Authorizing `identity:create_grant(project_id=d9d68d310bf24449a5fe20d2dc264a58, user_id=2b23f4478f49471fba6f43e6e4be99da, role_id=c9ce798acd4b482f8f391c6af97eb7a4)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:50.851937 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f074ca-c261-43d5-a592-dfa0304dce74 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:50.863637 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:50.868382 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 318/631] 10.222.0.26 () {58 vars in 1439 bytes} [Sat Aug 6 01:20:50 2022] PUT /identity/v3/projects/d9d68d310bf24449a5fe20d2dc264a58/users/2b23f4478f49471fba6f43e6e4be99da/roles/c9ce798acd4b482f8f391c6af97eb7a4 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:20:50.869634 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f074ca-c261-43d5-a592-dfa0304dce74 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:50.943027 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0085aeb1-3ee0-408c-a506-28013af5c749 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:50.943804 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:50.988792 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c724e5-fb8e-4b48-b9ad-31863585289b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:50.989548 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c724e5-fb8e-4b48-b9ad-31863585289b None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:50.989990 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c724e5-fb8e-4b48-b9ad-31863585289b None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:50.990369 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c724e5-fb8e-4b48-b9ad-31863585289b None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:50.991211 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c724e5-fb8e-4b48-b9ad-31863585289b None admin] RBAC: Adding query filter params (name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:50.991623 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c724e5-fb8e-4b48-b9ad-31863585289b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:50.993151 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c724e5-fb8e-4b48-b9ad-31863585289b None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:51.003477 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 314/632] 10.222.0.26 () {56 vars in 1103 bytes} [Sat Aug 6 01:20:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:51.005130 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75f2eef-04e8-44f3-a998-df8d0ef7d7b4 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:51.005487 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75f2eef-04e8-44f3-a998-df8d0ef7d7b4 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:51.005913 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75f2eef-04e8-44f3-a998-df8d0ef7d7b4 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:51.011843 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 315/633] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:20:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:20:51.012046 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f074ca-c261-43d5-a592-dfa0304dce74 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:51.012608 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75f2eef-04e8-44f3-a998-df8d0ef7d7b4 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:51.057651 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb66976-e0ce-4f60-8500-57c56916e401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:51.059013 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb66976-e0ce-4f60-8500-57c56916e401 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:51.059336 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb66976-e0ce-4f60-8500-57c56916e401 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:51.059656 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb66976-e0ce-4f60-8500-57c56916e401 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:51.137513 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb66976-e0ce-4f60-8500-57c56916e401 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:51.138984 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb66976-e0ce-4f60-8500-57c56916e401 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:51.153041 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75f2eef-04e8-44f3-a998-df8d0ef7d7b4 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:51.165494 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:51.192389 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0112f2c6-923c-4b2b-ad18-c03941753f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:51.193167 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0112f2c6-923c-4b2b-ad18-c03941753f2d None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:51.193522 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0112f2c6-923c-4b2b-ad18-c03941753f2d None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:51.193871 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0112f2c6-923c-4b2b-ad18-c03941753f2d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:51.205439 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0112f2c6-923c-4b2b-ad18-c03941753f2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:51.205821 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:51.207024 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0112f2c6-923c-4b2b-ad18-c03941753f2d None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:51.222880 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-0112f2c6-923c-4b2b-ad18-c03941753f2d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:20:51.222880 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:51.222880 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:51.222880 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:51.222880 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:51.222880 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:51.222880 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:51.222880 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:51.222880 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:51.222880 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:51.222880 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:51.222880 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:51.222880 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:20:51.222880 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:20:51.222880 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:20:51.223821 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:20:51.223821 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:51.223821 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:51.223821 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:51.223821 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:51.223821 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:51.223821 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:51.223821 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:51.223821 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:51.223821 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:51.223821 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:51.223821 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:51.223821 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:51.223821 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:51.223821 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:51.223821 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:51.223821 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:51.224701 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:20:51.224701 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:20:51.224701 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:20:51.224701 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:20:51.229145 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 316/634] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:20:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:20:51.231039 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d766f26e-18f5-452f-8dd6-8df4661bce31 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:51.231460 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d766f26e-18f5-452f-8dd6-8df4661bce31 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:51.231811 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d766f26e-18f5-452f-8dd6-8df4661bce31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:51.254503 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-d766f26e-18f5-452f-8dd6-8df4661bce31 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:20:51.256116 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-d766f26e-18f5-452f-8dd6-8df4661bce31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:20:51.280318 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 319/635] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Aug 6 01:20:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 01:20:51.281892 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb66976-e0ce-4f60-8500-57c56916e401 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:51.349121 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 317/636] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:20:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:20:51.350347 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d766f26e-18f5-452f-8dd6-8df4661bce31 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:51.398296 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb66976-e0ce-4f60-8500-57c56916e401 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:51.439078 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa721a71-4582-400b-b444-c0f4e0653609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:51.439759 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa721a71-4582-400b-b444-c0f4e0653609 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:51.439985 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa721a71-4582-400b-b444-c0f4e0653609 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:51.440248 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa721a71-4582-400b-b444-c0f4e0653609 None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:51.441079 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa721a71-4582-400b-b444-c0f4e0653609 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:51.441768 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa721a71-4582-400b-b444-c0f4e0653609 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:51.442969 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa721a71-4582-400b-b444-c0f4e0653609 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:51.459536 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 320/637] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Aug 6 01:20:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:51.459536 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a04e7ae-7ba9-49b4-a187-0a69d8425556 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:51.459536 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a04e7ae-7ba9-49b4-a187-0a69d8425556 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:51.459536 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a04e7ae-7ba9-49b4-a187-0a69d8425556 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:51.459536 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 321/638] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:20:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:20:51.459536 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a04e7ae-7ba9-49b4-a187-0a69d8425556 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:51.470279 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:51.472030 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:51.482766 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d766f26e-18f5-452f-8dd6-8df4661bce31 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:51.521289 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7311aaf2-faa4-4e54-8254-23e79f236f3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:51.521973 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7311aaf2-faa4-4e54-8254-23e79f236f3b None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:51.522258 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7311aaf2-faa4-4e54-8254-23e79f236f3b None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:51.522539 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7311aaf2-faa4-4e54-8254-23e79f236f3b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:51.532886 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7311aaf2-faa4-4e54-8254-23e79f236f3b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:51.534390 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7311aaf2-faa4-4e54-8254-23e79f236f3b None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:51.552157 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-7311aaf2-faa4-4e54-8254-23e79f236f3b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:20:51.552157 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:51.552157 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:51.552157 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:51.552157 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:51.552157 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:51.552157 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:51.552157 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:51.552157 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:51.552157 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:51.552157 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:51.552157 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:51.552157 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:20:51.552157 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:20:51.552157 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:20:51.553152 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:20:51.553152 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:51.553152 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:51.553152 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:51.553152 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:51.553152 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:51.553152 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:51.553152 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:51.553152 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:51.553152 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:51.553152 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:51.553152 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:51.553152 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:51.553152 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:51.553152 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:51.553152 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:51.553152 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:51.554054 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:20:51.554054 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:20:51.554054 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:20:51.554054 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:20:51.554054 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 318/639] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:20:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:20:51.554616 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7311aaf2-faa4-4e54-8254-23e79f236f3b None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:51.577726 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a04e7ae-7ba9-49b4-a187-0a69d8425556 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:51.615304 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87abda8f-1de2-4b4a-8bf4-0cfa08832344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:51.616152 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87abda8f-1de2-4b4a-8bf4-0cfa08832344 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:51.616262 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87abda8f-1de2-4b4a-8bf4-0cfa08832344 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:51.616480 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87abda8f-1de2-4b4a-8bf4-0cfa08832344 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:51.627906 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87abda8f-1de2-4b4a-8bf4-0cfa08832344 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:20:51.633322 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-87abda8f-1de2-4b4a-8bf4-0cfa08832344 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 06 01:20:51.633322 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:51.633322 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:51.633322 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:51.633322 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:51.633322 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:51.633322 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:51.633322 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:51.633322 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:51.633322 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:51.633322 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:51.633322 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:51.633322 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:20:51.633322 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:20:51.633322 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:20:51.633322 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:20:51.634243 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:20:51.634243 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:20:51.634243 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:20:51.634243 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:20:51.634243 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:51.634243 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:51.634243 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:51.634243 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:51.634243 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:51.634243 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:51.634243 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:51.634243 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:51.634243 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:51.634243 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:51.634243 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:51.634243 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:51.635092 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:51.635092 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:51.635092 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:51.635092 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:51.635092 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:20:51.635092 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:20:51.635092 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:20:51.635092 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:20:51.635092 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:20:51.635092 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:20:51.635092 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 06 01:20:51.635092 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:20:51.635092 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 322/640] 10.222.0.26 () {56 vars in 1102 bytes} [Sat Aug 6 01:20:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:20:51.635818 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bc9830-4175-4986-9437-a5b9d194eb38 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:51.635887 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bc9830-4175-4986-9437-a5b9d194eb38 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:51.637779 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bc9830-4175-4986-9437-a5b9d194eb38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:51.656518 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-b4bc9830-4175-4986-9437-a5b9d194eb38 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:20:51.657610 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-b4bc9830-4175-4986-9437-a5b9d194eb38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:20:51.672205 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7311aaf2-faa4-4e54-8254-23e79f236f3b None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:51.706565 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7add96a-798e-4d24-ae55-289f8af2b196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:51.707205 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7add96a-798e-4d24-ae55-289f8af2b196 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:51.707492 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7add96a-798e-4d24-ae55-289f8af2b196 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:51.707769 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7add96a-798e-4d24-ae55-289f8af2b196 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:51.716368 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7add96a-798e-4d24-ae55-289f8af2b196 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:51.717829 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7add96a-798e-4d24-ae55-289f8af2b196 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:51.728159 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:51.729752 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-c7add96a-798e-4d24-ae55-289f8af2b196 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 06 01:20:51.729752 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:51.729752 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:51.729752 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:51.729752 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:51.729752 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:51.729752 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:51.729752 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:51.729752 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:51.729752 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:51.729752 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:51.729752 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:51.729752 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 01:20:51.729752 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 01:20:51.729752 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 01:20:51.729752 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 01:20:51.731350 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:51.731350 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:51.731350 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 01:20:51.731350 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:20:51.731350 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 01:20:51.731350 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:20:51.731350 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:51.731350 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:51.731350 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:51.731350 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:51.731350 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:51.731350 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:51.731350 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:51.731350 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:51.731350 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:51.731350 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:51.731350 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:51.731350 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:51.732828 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:51.732828 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:51.732828 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 01:20:51.732828 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 01:20:51.732828 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 01:20:51.732828 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 01:20:51.732828 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 01:20:51.732828 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 01:20:51.732828 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 06 01:20:51.732828 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:20:51.732828 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 319/641] 10.222.0.26 () {56 vars in 1093 bytes} [Sat Aug 6 01:20:51 2022] GET /identity/v3/users/nova => generated 82 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:20:51.732828 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7add96a-798e-4d24-ae55-289f8af2b196 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:51.737712 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:51.738877 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 323/642] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:20:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:20:51.739577 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bc9830-4175-4986-9437-a5b9d194eb38 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:51.844486 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7add96a-798e-4d24-ae55-289f8af2b196 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:51.845688 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bc9830-4175-4986-9437-a5b9d194eb38 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:51.879711 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a830a4f-3efd-4c90-b769-da6edb8fe9b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:51.880329 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a830a4f-3efd-4c90-b769-da6edb8fe9b6 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:51.880608 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a830a4f-3efd-4c90-b769-da6edb8fe9b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:51.880881 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a830a4f-3efd-4c90-b769-da6edb8fe9b6 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:51.881598 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a830a4f-3efd-4c90-b769-da6edb8fe9b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:51.881900 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a830a4f-3efd-4c90-b769-da6edb8fe9b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:51.882010 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe45920a-b1f0-4cf5-a6cc-9faabcf5a313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:51.882603 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe45920a-b1f0-4cf5-a6cc-9faabcf5a313 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:51.882885 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe45920a-b1f0-4cf5-a6cc-9faabcf5a313 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:51.883161 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe45920a-b1f0-4cf5-a6cc-9faabcf5a313 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:51.883253 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a830a4f-3efd-4c90-b769-da6edb8fe9b6 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:51.883874 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe45920a-b1f0-4cf5-a6cc-9faabcf5a313 None admin] RBAC: Adding query filter params (name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:51.884177 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe45920a-b1f0-4cf5-a6cc-9faabcf5a313 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:51.885638 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe45920a-b1f0-4cf5-a6cc-9faabcf5a313 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:51.894462 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 324/643] 10.222.0.26 () {56 vars in 1103 bytes} [Sat Aug 6 01:20:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:51.895092 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe45920a-b1f0-4cf5-a6cc-9faabcf5a313 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:51.900181 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 320/644] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Aug 6 01:20:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:51.901256 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a830a4f-3efd-4c90-b769-da6edb8fe9b6 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:51.956519 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:51.978989 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:52.001422 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe45920a-b1f0-4cf5-a6cc-9faabcf5a313 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:52.022288 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a830a4f-3efd-4c90-b769-da6edb8fe9b6 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:52.044341 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8890cd57-d88f-446f-8bcb-2adae02e0066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:52.044965 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8890cd57-d88f-446f-8bcb-2adae02e0066 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:52.045283 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8890cd57-d88f-446f-8bcb-2adae02e0066 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:52.045568 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8890cd57-d88f-446f-8bcb-2adae02e0066 None admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:52.046365 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8890cd57-d88f-446f-8bcb-2adae02e0066 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:52.046678 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8890cd57-d88f-446f-8bcb-2adae02e0066 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:52.048117 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8890cd57-d88f-446f-8bcb-2adae02e0066 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:52.061023 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abc075c-0aec-48e6-b300-39a776c30d24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:52.061920 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abc075c-0aec-48e6-b300-39a776c30d24 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:52.062843 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abc075c-0aec-48e6-b300-39a776c30d24 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:52.063321 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abc075c-0aec-48e6-b300-39a776c30d24 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:52.070080 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 325/645] 10.222.0.26 () {56 vars in 1133 bytes} [Sat Aug 6 01:20:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:52.070080 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8890cd57-d88f-446f-8bcb-2adae02e0066 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:52.079986 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abc075c-0aec-48e6-b300-39a776c30d24 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:20:52.086725 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-5abc075c-0aec-48e6-b300-39a776c30d24 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 06 01:20:52.086725 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:52.086725 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:52.086725 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:52.086725 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:52.086725 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:52.086725 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:52.086725 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:52.086725 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:52.086725 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:52.086725 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:52.086725 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:52.086725 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:20:52.086725 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:20:52.086725 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:20:52.086725 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:20:52.087824 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:20:52.087824 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:20:52.087824 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:20:52.087824 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:20:52.087824 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:52.087824 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:52.087824 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:52.087824 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:52.087824 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:52.087824 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:52.087824 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:52.087824 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:52.087824 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:52.087824 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:52.087824 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:52.087824 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:52.088812 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:52.088812 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:52.088812 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:52.088812 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:52.088812 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:20:52.088812 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:20:52.088812 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:20:52.088812 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:20:52.088812 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:20:52.088812 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:20:52.088812 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 06 01:20:52.088812 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:20:52.088812 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 321/646] 10.222.0.26 () {56 vars in 1102 bytes} [Sat Aug 6 01:20:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:20:52.090318 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abc075c-0aec-48e6-b300-39a776c30d24 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:52.187872 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8890cd57-d88f-446f-8bcb-2adae02e0066 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:52.203375 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abc075c-0aec-48e6-b300-39a776c30d24 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:52.209453 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:52.225072 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2687183a-a14f-4184-b76c-3c89d615fae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:52.225740 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2687183a-a14f-4184-b76c-3c89d615fae3 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:52.226261 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2687183a-a14f-4184-b76c-3c89d615fae3 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:52.226636 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2687183a-a14f-4184-b76c-3c89d615fae3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:52.229488 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:52.237906 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2687183a-a14f-4184-b76c-3c89d615fae3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:52.239341 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2687183a-a14f-4184-b76c-3c89d615fae3 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:52.242330 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5176743d-99ec-4a5d-9713-d07ce6d94dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:52.242960 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5176743d-99ec-4a5d-9713-d07ce6d94dcd None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:52.243242 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5176743d-99ec-4a5d-9713-d07ce6d94dcd None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:52.243518 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5176743d-99ec-4a5d-9713-d07ce6d94dcd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:52.251610 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5176743d-99ec-4a5d-9713-d07ce6d94dcd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:52.253016 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5176743d-99ec-4a5d-9713-d07ce6d94dcd None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:52.254552 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-2687183a-a14f-4184-b76c-3c89d615fae3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:20:52.254552 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:52.254552 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:52.254552 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:52.254552 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:52.254552 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:52.254552 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:52.254552 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:52.254552 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:52.254552 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:52.254552 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:52.254552 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:52.254552 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:20:52.254552 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:20:52.254552 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:20:52.255471 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:20:52.255471 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:52.255471 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:52.255471 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:52.255471 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:52.255471 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:52.255471 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:52.255471 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:52.255471 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:52.255471 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:52.255471 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:52.255471 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:52.255471 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:52.255471 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:52.255471 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:52.255471 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:52.255471 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:52.256419 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:20:52.256419 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:20:52.256419 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:20:52.256419 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:20:52.256419 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 326/647] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:20:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:20:52.256723 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2687183a-a14f-4184-b76c-3c89d615fae3 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:52.270367 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-5176743d-99ec-4a5d-9713-d07ce6d94dcd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:20:52.270367 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:52.270367 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:52.270367 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:52.270367 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:52.270367 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:52.270367 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:52.270367 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:52.270367 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:52.270367 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:52.270367 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:52.270367 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:52.270367 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 01:20:52.270367 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 01:20:52.270367 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 01:20:52.271308 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 01:20:52.271308 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:52.271308 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:52.271308 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:52.271308 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:52.271308 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:52.271308 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:52.271308 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:52.271308 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:52.271308 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:52.271308 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:52.271308 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:52.271308 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:52.271308 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:52.271308 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:52.271308 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:52.271308 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:52.272192 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 01:20:52.272192 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 01:20:52.272192 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 01:20:52.272192 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 01:20:52.272192 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 01:20:52.272192 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 01:20:52.272192 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:20:52.272192 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:20:52.276361 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 322/648] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Aug 6 01:20:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:20:52.277304 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5176743d-99ec-4a5d-9713-d07ce6d94dcd None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:52.369887 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2687183a-a14f-4184-b76c-3c89d615fae3 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:52.383319 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5176743d-99ec-4a5d-9713-d07ce6d94dcd None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:52.414720 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29487cb6-0bf2-4923-9c9e-418fb758d436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:52.415343 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29487cb6-0bf2-4923-9c9e-418fb758d436 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:52.415625 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29487cb6-0bf2-4923-9c9e-418fb758d436 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:52.415900 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29487cb6-0bf2-4923-9c9e-418fb758d436 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:52.426389 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e51fc5-8cb9-43ce-afce-941221804b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:52.427254 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e51fc5-8cb9-43ce-afce-941221804b33 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:52.427540 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e51fc5-8cb9-43ce-afce-941221804b33 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:52.427822 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e51fc5-8cb9-43ce-afce-941221804b33 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:52.428538 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e51fc5-8cb9-43ce-afce-941221804b33 None admin] RBAC: Adding query filter params (name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:52.428986 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e51fc5-8cb9-43ce-afce-941221804b33 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:52.430330 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e51fc5-8cb9-43ce-afce-941221804b33 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:52.439597 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 323/649] 10.222.0.26 () {56 vars in 1103 bytes} [Sat Aug 6 01:20:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:52.440158 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e51fc5-8cb9-43ce-afce-941221804b33 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:52.465745 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:52.491078 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:52.497022 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29487cb6-0bf2-4923-9c9e-418fb758d436 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:52.498532 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29487cb6-0bf2-4923-9c9e-418fb758d436 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:52.549676 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e51fc5-8cb9-43ce-afce-941221804b33 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:52.590674 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76b4bab-cbe1-4234-8c0d-a263a8ded4c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:52.591443 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76b4bab-cbe1-4234-8c0d-a263a8ded4c0 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:52.591719 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76b4bab-cbe1-4234-8c0d-a263a8ded4c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:52.591997 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76b4bab-cbe1-4234-8c0d-a263a8ded4c0 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:52.592682 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76b4bab-cbe1-4234-8c0d-a263a8ded4c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:52.592983 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76b4bab-cbe1-4234-8c0d-a263a8ded4c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:52.594382 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76b4bab-cbe1-4234-8c0d-a263a8ded4c0 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:52.603063 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 327/650] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Aug 6 01:20:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 01:20:52.610987 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 324/651] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Aug 6 01:20:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:52.610987 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76b4bab-cbe1-4234-8c0d-a263a8ded4c0 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:52.611257 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29487cb6-0bf2-4923-9c9e-418fb758d436 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:52.684360 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:52.717277 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29487cb6-0bf2-4923-9c9e-418fb758d436 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:52.720595 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76b4bab-cbe1-4234-8c0d-a263a8ded4c0 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:52.728833 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:52.755021 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626bf8d5-5e50-4ea8-97ed-43033dc3fa53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:52.755670 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626bf8d5-5e50-4ea8-97ed-43033dc3fa53 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:52.755919 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626bf8d5-5e50-4ea8-97ed-43033dc3fa53 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:52.756193 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626bf8d5-5e50-4ea8-97ed-43033dc3fa53 None admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:52.756983 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626bf8d5-5e50-4ea8-97ed-43033dc3fa53 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:52.757272 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626bf8d5-5e50-4ea8-97ed-43033dc3fa53 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:52.758121 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9ba1cb-6bca-46a5-8a5c-fc8196cde9c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:52.758722 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626bf8d5-5e50-4ea8-97ed-43033dc3fa53 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:52.758827 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9ba1cb-6bca-46a5-8a5c-fc8196cde9c5 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:52.759029 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9ba1cb-6bca-46a5-8a5c-fc8196cde9c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:52.759306 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9ba1cb-6bca-46a5-8a5c-fc8196cde9c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:52.770559 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9ba1cb-6bca-46a5-8a5c-fc8196cde9c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:52.776254 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 328/652] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Aug 6 01:20:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:52.776401 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9ba1cb-6bca-46a5-8a5c-fc8196cde9c5 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:52.776487 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626bf8d5-5e50-4ea8-97ed-43033dc3fa53 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:52.792939 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-3c9ba1cb-6bca-46a5-8a5c-fc8196cde9c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:20:52.792939 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:52.792939 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:52.792939 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:52.792939 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:52.792939 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:52.792939 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:52.792939 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:52.792939 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:52.792939 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:52.792939 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:52.792939 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:52.792939 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:20:52.792939 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:20:52.792939 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:20:52.793907 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:20:52.793907 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:52.793907 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:52.793907 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:52.793907 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:52.793907 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:52.793907 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:52.793907 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:52.793907 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:52.793907 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:52.793907 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:52.793907 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:52.793907 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:52.793907 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:52.793907 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:52.793907 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:52.793907 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:52.794768 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:20:52.794768 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:20:52.794768 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:20:52.794768 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:20:52.794768 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 325/653] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:20:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:20:52.795515 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9ba1cb-6bca-46a5-8a5c-fc8196cde9c5 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:52.889990 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626bf8d5-5e50-4ea8-97ed-43033dc3fa53 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:52.905693 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9ba1cb-6bca-46a5-8a5c-fc8196cde9c5 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:52.922491 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:52.928943 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba27687a-7c99-47ab-b819-1da01741beec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:52.929622 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba27687a-7c99-47ab-b819-1da01741beec None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:52.929903 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba27687a-7c99-47ab-b819-1da01741beec None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:52.930232 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba27687a-7c99-47ab-b819-1da01741beec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:52.941590 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba27687a-7c99-47ab-b819-1da01741beec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:52.942479 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00c9d8d-8240-41a3-b3d2-90b5f3541b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:52.943040 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba27687a-7c99-47ab-b819-1da01741beec None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:52.943143 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00c9d8d-8240-41a3-b3d2-90b5f3541b39 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:52.943399 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00c9d8d-8240-41a3-b3d2-90b5f3541b39 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:52.943671 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00c9d8d-8240-41a3-b3d2-90b5f3541b39 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:52.951310 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00c9d8d-8240-41a3-b3d2-90b5f3541b39 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:52.952734 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00c9d8d-8240-41a3-b3d2-90b5f3541b39 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:52.957309 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-ba27687a-7c99-47ab-b819-1da01741beec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:20:52.957309 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:52.957309 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:52.957309 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:52.957309 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:52.957309 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:52.957309 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:52.957309 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:52.957309 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:52.957309 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:52.957309 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:52.957309 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:52.957309 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:20:52.957309 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:20:52.957309 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:20:52.958282 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:20:52.958282 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:52.958282 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:52.958282 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:52.958282 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:52.958282 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:52.958282 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:52.958282 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:52.958282 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:52.958282 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:52.958282 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:52.958282 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:52.958282 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:52.958282 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:52.958282 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:52.958282 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:52.958282 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:52.963919 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:20:52.963919 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:20:52.963919 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:20:52.963919 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:20:52.963919 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 329/654] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:20:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:20:52.963919 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba27687a-7c99-47ab-b819-1da01741beec None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:52.964816 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-d00c9d8d-8240-41a3-b3d2-90b5f3541b39 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 06 01:20:52.964816 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:52.964816 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:52.964816 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:52.964816 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:52.964816 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:52.964816 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:52.964816 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:52.964816 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:52.964816 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:52.964816 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:52.964816 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:52.964816 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 01:20:52.964816 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 01:20:52.964816 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 01:20:52.964816 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 01:20:52.965712 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:52.965712 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:52.965712 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 01:20:52.965712 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:20:52.965712 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 01:20:52.965712 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:20:52.965712 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:52.965712 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:52.965712 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:52.965712 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:52.965712 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:52.965712 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:52.965712 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:52.965712 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:52.965712 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:52.965712 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:52.965712 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:52.965712 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:52.966642 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:52.966642 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:52.966642 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 01:20:52.966642 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 01:20:52.966642 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 01:20:52.966642 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 01:20:52.966642 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 01:20:52.966642 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 01:20:52.966642 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 06 01:20:52.966642 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:20:52.966642 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 326/655] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Aug 6 01:20:52 2022] GET /identity/v3/users/cinder => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:20:52.967264 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00c9d8d-8240-41a3-b3d2-90b5f3541b39 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:52.986448 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:53.064836 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba27687a-7c99-47ab-b819-1da01741beec None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:53.068297 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00c9d8d-8240-41a3-b3d2-90b5f3541b39 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:53.098843 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f8d90a-a3ba-4f31-a006-ec6c9cf22c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:53.099453 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f8d90a-a3ba-4f31-a006-ec6c9cf22c11 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:53.099737 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f8d90a-a3ba-4f31-a006-ec6c9cf22c11 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:53.100017 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f8d90a-a3ba-4f31-a006-ec6c9cf22c11 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:53.100801 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f8d90a-a3ba-4f31-a006-ec6c9cf22c11 None admin] RBAC: Adding query filter params (role.id=c9ce798acd4b482f8f391c6af97eb7a4, scope.project.id=d9d68d310bf24449a5fe20d2dc264a58, user.id=abb76a5c8ef74bf2b18c56074daaad58) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:53.101480 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f8d90a-a3ba-4f31-a006-ec6c9cf22c11 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:53.101592 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5551d31-97a6-436d-97bf-31550d52c8a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:53.102148 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5551d31-97a6-436d-97bf-31550d52c8a1 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:53.102430 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5551d31-97a6-436d-97bf-31550d52c8a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:53.102705 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5551d31-97a6-436d-97bf-31550d52c8a1 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:53.102900 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f8d90a-a3ba-4f31-a006-ec6c9cf22c11 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:53.103391 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5551d31-97a6-436d-97bf-31550d52c8a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:53.103697 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5551d31-97a6-436d-97bf-31550d52c8a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:53.105695 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5551d31-97a6-436d-97bf-31550d52c8a1 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:53.109242 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 330/656] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Aug 6 01:20:52 2022] GET /identity/v3/role_assignments?user.id=abb76a5c8ef74bf2b18c56074daaad58&scope.project.id=d9d68d310bf24449a5fe20d2dc264a58&role.id=c9ce798acd4b482f8f391c6af97eb7a4 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:53.110398 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008dfcf2-6247-4b22-b716-22a84718b8bc None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:53.110687 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008dfcf2-6247-4b22-b716-22a84718b8bc None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:53.110961 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008dfcf2-6247-4b22-b716-22a84718b8bc None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:53.112380 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 331/657] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:20:53 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:20:53.112850 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008dfcf2-6247-4b22-b716-22a84718b8bc None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:53.133379 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 327/658] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Aug 6 01:20:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:53.133379 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5551d31-97a6-436d-97bf-31550d52c8a1 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:53.181129 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:53.223377 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:53.244018 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008dfcf2-6247-4b22-b716-22a84718b8bc None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:53.250102 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5551d31-97a6-436d-97bf-31550d52c8a1 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:53.286654 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b2991f-0733-4c6b-a271-9be7f3a081bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:53.287286 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b2991f-0733-4c6b-a271-9be7f3a081bb None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:53.287573 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b2991f-0733-4c6b-a271-9be7f3a081bb None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:53.287858 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b2991f-0733-4c6b-a271-9be7f3a081bb None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:53.288556 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b2991f-0733-4c6b-a271-9be7f3a081bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:53.289605 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b2991f-0733-4c6b-a271-9be7f3a081bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:53.289736 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b271981-19c2-44e3-a195-4cfa45771689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:53.290041 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b271981-19c2-44e3-a195-4cfa45771689 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:53.290326 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b271981-19c2-44e3-a195-4cfa45771689 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:53.290420 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b2991f-0733-4c6b-a271-9be7f3a081bb None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:53.290606 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b271981-19c2-44e3-a195-4cfa45771689 None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:53.292087 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b271981-19c2-44e3-a195-4cfa45771689 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:53.292087 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b271981-19c2-44e3-a195-4cfa45771689 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:53.295247 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b271981-19c2-44e3-a195-4cfa45771689 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:53.304965 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 332/659] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Aug 6 01:20:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:53.306160 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b876b4-c0a6-4fe2-acce-938949e44896 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:53.306449 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b876b4-c0a6-4fe2-acce-938949e44896 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:53.306730 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b876b4-c0a6-4fe2-acce-938949e44896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:53.323125 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 328/660] 10.222.0.26 () {56 vars in 1137 bytes} [Sat Aug 6 01:20:53 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:53.324062 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b271981-19c2-44e3-a195-4cfa45771689 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:53.325731 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-73b876b4-c0a6-4fe2-acce-938949e44896 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:20:53.327130 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-73b876b4-c0a6-4fe2-acce-938949e44896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:20:53.405374 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 333/661] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:20:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:20:53.405539 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b876b4-c0a6-4fe2-acce-938949e44896 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:53.437951 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b271981-19c2-44e3-a195-4cfa45771689 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:53.460389 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:53.476910 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ef97f6-3b60-4373-9c8c-e067d0b79072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:53.477586 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ef97f6-3b60-4373-9c8c-e067d0b79072 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:53.477864 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ef97f6-3b60-4373-9c8c-e067d0b79072 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:53.478133 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ef97f6-3b60-4373-9c8c-e067d0b79072 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:53.481776 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:53.487683 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ef97f6-3b60-4373-9c8c-e067d0b79072 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:53.489668 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ef97f6-3b60-4373-9c8c-e067d0b79072 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:53.502714 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-44ef97f6-3b60-4373-9c8c-e067d0b79072 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 06 01:20:53.502714 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:53.502714 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:53.502714 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:53.502714 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:53.502714 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:53.502714 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:53.502714 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:53.502714 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:53.502714 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:53.502714 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:53.502714 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:53.502714 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 01:20:53.502714 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 01:20:53.502714 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 01:20:53.502714 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 01:20:53.503674 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:53.503674 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:53.503674 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 01:20:53.503674 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:20:53.503674 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 01:20:53.503674 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:20:53.503674 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:53.503674 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:53.503674 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:53.503674 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:53.503674 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:53.503674 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:53.503674 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:53.503674 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:53.503674 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:53.503674 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:53.503674 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:53.503674 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:53.504604 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:53.504604 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:53.504604 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 01:20:53.504604 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 01:20:53.504604 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 01:20:53.504604 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 01:20:53.504604 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 01:20:53.504604 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 01:20:53.504604 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 06 01:20:53.504604 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:20:53.504604 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 329/662] 10.222.0.26 () {56 vars in 1102 bytes} [Sat Aug 6 01:20:53 2022] GET /identity/v3/users/neutron => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:20:53.504604 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ef97f6-3b60-4373-9c8c-e067d0b79072 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:53.531744 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b876b4-c0a6-4fe2-acce-938949e44896 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:53.566775 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe15eca1-ee84-4732-950c-c0fdadd81689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:53.567820 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe15eca1-ee84-4732-950c-c0fdadd81689 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:53.568100 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe15eca1-ee84-4732-950c-c0fdadd81689 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:53.568518 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe15eca1-ee84-4732-950c-c0fdadd81689 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:53.577291 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe15eca1-ee84-4732-950c-c0fdadd81689 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:53.579060 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe15eca1-ee84-4732-950c-c0fdadd81689 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:53.593291 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-fe15eca1-ee84-4732-950c-c0fdadd81689 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:20:53.593291 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:53.593291 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:53.593291 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:53.593291 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:53.593291 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:53.593291 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:53.593291 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:53.593291 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:53.593291 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:53.593291 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:53.593291 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:53.593291 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 01:20:53.593291 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 01:20:53.593291 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 01:20:53.594177 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 01:20:53.594177 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:53.594177 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:53.594177 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:53.594177 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:53.594177 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:53.594177 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:53.594177 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:53.594177 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:53.594177 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:53.594177 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:53.594177 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:53.594177 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:53.594177 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:53.594177 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:53.594177 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:53.594177 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:53.595144 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 01:20:53.595144 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 01:20:53.595144 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 01:20:53.595144 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 01:20:53.595144 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 01:20:53.595144 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 01:20:53.595144 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:20:53.595144 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:20:53.596215 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 334/663] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Aug 6 01:20:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:20:53.596215 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe15eca1-ee84-4732-950c-c0fdadd81689 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:53.621597 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ef97f6-3b60-4373-9c8c-e067d0b79072 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:53.660147 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae493f81-f506-4278-ab2b-af28b874ad3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:53.660773 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae493f81-f506-4278-ab2b-af28b874ad3d None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:53.661079 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae493f81-f506-4278-ab2b-af28b874ad3d None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:53.661381 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae493f81-f506-4278-ab2b-af28b874ad3d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:53.703660 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:53.708342 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe15eca1-ee84-4732-950c-c0fdadd81689 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:53.730963 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:53.739964 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae493f81-f506-4278-ab2b-af28b874ad3d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:53.741598 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae493f81-f506-4278-ab2b-af28b874ad3d None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:53.744988 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f25fca-6988-4bd2-981f-ff1135a9aedb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:53.745705 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f25fca-6988-4bd2-981f-ff1135a9aedb None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:53.745986 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f25fca-6988-4bd2-981f-ff1135a9aedb None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:53.746258 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f25fca-6988-4bd2-981f-ff1135a9aedb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:53.757761 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f25fca-6988-4bd2-981f-ff1135a9aedb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:20:53.763064 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-f7f25fca-6988-4bd2-981f-ff1135a9aedb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 06 01:20:53.763064 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:53.763064 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:53.763064 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:53.763064 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:53.763064 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:53.763064 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:53.763064 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:53.763064 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:53.763064 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:53.763064 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:53.763064 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:53.763064 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:20:53.763064 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:20:53.763064 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:20:53.763064 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:20:53.763993 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:20:53.763993 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:20:53.763993 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:20:53.763993 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:20:53.763993 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:53.763993 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:53.763993 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:53.763993 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:53.763993 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:53.763993 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:53.763993 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:53.763993 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:53.763993 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:53.763993 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:53.763993 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:53.763993 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:53.764854 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:53.764854 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:53.764854 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:53.764854 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:53.764854 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:20:53.764854 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:20:53.764854 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:20:53.764854 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:20:53.764854 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:20:53.764854 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:20:53.764854 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 06 01:20:53.764854 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:20:53.764854 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 335/664] 10.222.0.26 () {56 vars in 1102 bytes} [Sat Aug 6 01:20:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:20:53.765631 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f25fca-6988-4bd2-981f-ff1135a9aedb None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:53.849792 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 330/665] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Aug 6 01:20:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 01:20:53.854851 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae493f81-f506-4278-ab2b-af28b874ad3d None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:53.879251 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f25fca-6988-4bd2-981f-ff1135a9aedb None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:53.915965 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc98556-d224-42f7-a1fe-a13a57a7fb7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:53.916592 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc98556-d224-42f7-a1fe-a13a57a7fb7f None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:53.916870 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc98556-d224-42f7-a1fe-a13a57a7fb7f None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:53.917222 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc98556-d224-42f7-a1fe-a13a57a7fb7f None admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:53.918103 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc98556-d224-42f7-a1fe-a13a57a7fb7f None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:53.918409 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc98556-d224-42f7-a1fe-a13a57a7fb7f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:53.919833 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc98556-d224-42f7-a1fe-a13a57a7fb7f None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:53.938672 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 336/666] 10.222.0.26 () {56 vars in 1139 bytes} [Sat Aug 6 01:20:53 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:53.939488 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc98556-d224-42f7-a1fe-a13a57a7fb7f None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:53.957808 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:53.959320 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:53.967880 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae493f81-f506-4278-ab2b-af28b874ad3d None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:54.005002 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd60fec-6399-4d8e-9d5a-1ea638580936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:54.005774 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd60fec-6399-4d8e-9d5a-1ea638580936 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:54.006160 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd60fec-6399-4d8e-9d5a-1ea638580936 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:54.006532 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd60fec-6399-4d8e-9d5a-1ea638580936 None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:54.007381 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd60fec-6399-4d8e-9d5a-1ea638580936 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:54.007913 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd60fec-6399-4d8e-9d5a-1ea638580936 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:54.009377 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd60fec-6399-4d8e-9d5a-1ea638580936 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:54.026324 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 331/667] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Aug 6 01:20:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:54.026324 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd60fec-6399-4d8e-9d5a-1ea638580936 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:54.051421 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc98556-d224-42f7-a1fe-a13a57a7fb7f None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:54.088482 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4410df29-da7e-4dca-909f-02f53d4f05c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:54.089942 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4410df29-da7e-4dca-909f-02f53d4f05c2 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:54.090260 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4410df29-da7e-4dca-909f-02f53d4f05c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:54.090537 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4410df29-da7e-4dca-909f-02f53d4f05c2 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:54.091253 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4410df29-da7e-4dca-909f-02f53d4f05c2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:54.091556 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4410df29-da7e-4dca-909f-02f53d4f05c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:54.093245 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4410df29-da7e-4dca-909f-02f53d4f05c2 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:54.105332 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 337/668] 10.222.0.26 () {56 vars in 1103 bytes} [Sat Aug 6 01:20:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:54.106520 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4410df29-da7e-4dca-909f-02f53d4f05c2 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:54.133358 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd60fec-6399-4d8e-9d5a-1ea638580936 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:54.166740 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9d2694-42f3-43fb-9988-38e37fd37134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:54.168639 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9d2694-42f3-43fb-9988-38e37fd37134 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:54.169025 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9d2694-42f3-43fb-9988-38e37fd37134 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:54.169335 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9d2694-42f3-43fb-9988-38e37fd37134 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:54.179972 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9d2694-42f3-43fb-9988-38e37fd37134 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:54.181424 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9d2694-42f3-43fb-9988-38e37fd37134 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:54.192144 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:54.197705 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-df9d2694-42f3-43fb-9988-38e37fd37134 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:20:54.197705 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:54.197705 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:54.197705 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:54.197705 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:54.197705 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:54.197705 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:54.197705 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:54.197705 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:54.197705 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:54.197705 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:54.197705 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:54.197705 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:20:54.197705 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:20:54.197705 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:20:54.198643 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:20:54.198643 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:54.198643 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:54.198643 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:54.198643 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:54.198643 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:54.198643 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:54.198643 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:54.198643 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:54.198643 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:54.198643 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:54.198643 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:54.198643 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:54.198643 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:54.198643 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:54.198643 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:54.198643 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:54.199523 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:20:54.199523 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:20:54.199523 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:20:54.199523 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:20:54.199908 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 332/669] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:20:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:20:54.200836 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9d2694-42f3-43fb-9988-38e37fd37134 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:54.214538 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:54.222607 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4410df29-da7e-4dca-909f-02f53d4f05c2 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:54.263053 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a484bd9-6a4a-4d8d-ae74-cba9d9521a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:54.263604 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a484bd9-6a4a-4d8d-ae74-cba9d9521a64 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:54.263886 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a484bd9-6a4a-4d8d-ae74-cba9d9521a64 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:54.264157 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a484bd9-6a4a-4d8d-ae74-cba9d9521a64 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:54.318153 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9d2694-42f3-43fb-9988-38e37fd37134 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:54.345886 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a484bd9-6a4a-4d8d-ae74-cba9d9521a64 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:54.347332 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a484bd9-6a4a-4d8d-ae74-cba9d9521a64 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:54.353234 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8e60b2-8029-4260-85fa-f681c6f18af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:54.353855 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8e60b2-8029-4260-85fa-f681c6f18af1 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:54.354136 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8e60b2-8029-4260-85fa-f681c6f18af1 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:54.354410 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8e60b2-8029-4260-85fa-f681c6f18af1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:54.355761 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8e60b2-8029-4260-85fa-f681c6f18af1 None admin] RBAC: Adding query filter params (role.id=c9ce798acd4b482f8f391c6af97eb7a4, scope.project.id=d9d68d310bf24449a5fe20d2dc264a58, user.id=b87adc1d5bc240e99f95f6595b350be4) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:54.356159 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8e60b2-8029-4260-85fa-f681c6f18af1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:54.357606 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8e60b2-8029-4260-85fa-f681c6f18af1 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:54.373083 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 333/670] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Aug 6 01:20:54 2022] GET /identity/v3/role_assignments?user.id=b87adc1d5bc240e99f95f6595b350be4&scope.project.id=d9d68d310bf24449a5fe20d2dc264a58&role.id=c9ce798acd4b482f8f391c6af97eb7a4 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:54.373083 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8e60b2-8029-4260-85fa-f681c6f18af1 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:54.454102 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:54.463215 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:54.494973 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8e60b2-8029-4260-85fa-f681c6f18af1 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:54.498344 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 338/671] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Aug 6 01:20:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 01:20:54.499356 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a484bd9-6a4a-4d8d-ae74-cba9d9521a64 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:54.535641 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d0d7e1-320a-472d-a721-368d9039c0d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:54.536377 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d0d7e1-320a-472d-a721-368d9039c0d1 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:54.536753 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d0d7e1-320a-472d-a721-368d9039c0d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:54.537308 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d0d7e1-320a-472d-a721-368d9039c0d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:54.550281 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d0d7e1-320a-472d-a721-368d9039c0d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:54.551865 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d0d7e1-320a-472d-a721-368d9039c0d1 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:54.568445 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-92d0d7e1-320a-472d-a721-368d9039c0d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:20:54.568445 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:54.568445 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:54.568445 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:54.568445 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:54.568445 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:54.568445 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:54.568445 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:54.568445 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:54.568445 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:54.568445 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:54.568445 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:54.568445 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:20:54.568445 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:20:54.568445 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:20:54.569569 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:20:54.569569 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:54.569569 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:54.569569 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:54.569569 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:54.569569 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:54.569569 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:54.569569 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:54.569569 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:54.569569 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:54.569569 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:54.569569 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:54.569569 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:54.569569 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:54.569569 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:54.569569 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:54.569569 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:54.570687 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:20:54.570687 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:20:54.570687 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:20:54.570687 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:20:54.570687 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 334/672] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:20:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:20:54.571018 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d0d7e1-320a-472d-a721-368d9039c0d1 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:54.626257 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a484bd9-6a4a-4d8d-ae74-cba9d9521a64 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:54.672821 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e8a00f-607f-472f-bc8f-50cb225d7fd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:54.673644 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e8a00f-607f-472f-bc8f-50cb225d7fd2 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:54.674022 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e8a00f-607f-472f-bc8f-50cb225d7fd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:54.674402 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e8a00f-607f-472f-bc8f-50cb225d7fd2 None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:54.675206 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e8a00f-607f-472f-bc8f-50cb225d7fd2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:54.675608 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e8a00f-607f-472f-bc8f-50cb225d7fd2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:54.677125 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e8a00f-607f-472f-bc8f-50cb225d7fd2 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:54.690548 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 339/673] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Aug 6 01:20:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:54.691251 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e8a00f-607f-472f-bc8f-50cb225d7fd2 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:54.697607 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d0d7e1-320a-472d-a721-368d9039c0d1 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:54.721721 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:54.737522 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc66390f-f696-426f-9883-d26a3ac890c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:54.738371 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc66390f-f696-426f-9883-d26a3ac890c2 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:54.738760 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc66390f-f696-426f-9883-d26a3ac890c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:54.739137 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc66390f-f696-426f-9883-d26a3ac890c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:54.743100 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:54.816715 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e8a00f-607f-472f-bc8f-50cb225d7fd2 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:54.856173 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c64d4ea-a0ef-4e13-9e31-a9dced862d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:54.856835 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c64d4ea-a0ef-4e13-9e31-a9dced862d28 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:54.857163 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c64d4ea-a0ef-4e13-9e31-a9dced862d28 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:54.857441 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c64d4ea-a0ef-4e13-9e31-a9dced862d28 None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:54.858148 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c64d4ea-a0ef-4e13-9e31-a9dced862d28 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:54.858454 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c64d4ea-a0ef-4e13-9e31-a9dced862d28 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:54.859850 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c64d4ea-a0ef-4e13-9e31-a9dced862d28 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:54.878049 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 340/674] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Aug 6 01:20:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:54.878049 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c64d4ea-a0ef-4e13-9e31-a9dced862d28 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:54.913679 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc66390f-f696-426f-9883-d26a3ac890c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:54.916402 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc66390f-f696-426f-9883-d26a3ac890c2 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:54.937468 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-cc66390f-f696-426f-9883-d26a3ac890c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:20:54.937468 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:54.937468 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:54.937468 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:54.937468 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:54.937468 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:54.937468 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:54.937468 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:54.937468 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:54.937468 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:54.937468 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:54.937468 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:54.937468 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:20:54.937468 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:20:54.937468 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:20:54.938405 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:20:54.938405 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:54.938405 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:54.938405 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:54.938405 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:54.938405 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:54.938405 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:54.938405 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:54.938405 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:54.938405 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:54.938405 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:54.938405 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:54.938405 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:54.938405 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:54.938405 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:54.938405 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:54.938405 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:54.939275 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:20:54.939275 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:20:54.939275 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:20:54.939275 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:20:54.946095 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 335/675] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:20:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 371 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:20:54.946095 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc66390f-f696-426f-9883-d26a3ac890c2 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:55.001921 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:55.007946 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c64d4ea-a0ef-4e13-9e31-a9dced862d28 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:55.060017 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43522018-e7ed-471f-9470-36c2465a7aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:55.060663 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43522018-e7ed-471f-9470-36c2465a7aa3 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:55.060943 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43522018-e7ed-471f-9470-36c2465a7aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:55.061246 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43522018-e7ed-471f-9470-36c2465a7aa3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:55.063839 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc66390f-f696-426f-9883-d26a3ac890c2 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:55.073462 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43522018-e7ed-471f-9470-36c2465a7aa3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:55.076082 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43522018-e7ed-471f-9470-36c2465a7aa3 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:55.094105 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-43522018-e7ed-471f-9470-36c2465a7aa3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:20:55.094105 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:55.094105 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:55.094105 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:55.094105 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:55.094105 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:55.094105 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:55.094105 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:55.094105 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:55.094105 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:55.094105 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:55.094105 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:55.094105 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 01:20:55.094105 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 01:20:55.094105 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 01:20:55.095630 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 01:20:55.095630 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:55.095630 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:55.095630 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:55.095630 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:55.095630 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:55.095630 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:55.095630 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:55.095630 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:55.095630 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:55.095630 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:55.095630 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:55.095630 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:55.095630 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:55.095630 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:55.095630 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:55.095630 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:55.096704 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 01:20:55.096704 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 01:20:55.096704 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 01:20:55.096704 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 01:20:55.096704 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 01:20:55.096704 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 01:20:55.096704 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:20:55.096704 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:20:55.099352 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 341/676] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Aug 6 01:20:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:20:55.101456 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43522018-e7ed-471f-9470-36c2465a7aa3 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:55.105613 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafba1ca-8df0-4ea3-846d-339bb54ba149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:55.106372 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafba1ca-8df0-4ea3-846d-339bb54ba149 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:55.107416 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafba1ca-8df0-4ea3-846d-339bb54ba149 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:55.107416 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafba1ca-8df0-4ea3-846d-339bb54ba149 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:55.114924 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafba1ca-8df0-4ea3-846d-339bb54ba149 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:55.116455 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafba1ca-8df0-4ea3-846d-339bb54ba149 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:55.128906 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-cafba1ca-8df0-4ea3-846d-339bb54ba149 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 06 01:20:55.128906 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:55.128906 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:55.128906 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:55.128906 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:55.128906 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:55.128906 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:55.128906 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:55.128906 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:55.128906 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:55.128906 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:55.128906 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:55.128906 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 01:20:55.128906 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 01:20:55.128906 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 01:20:55.128906 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 01:20:55.130013 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:55.130013 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:55.130013 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 01:20:55.130013 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:20:55.130013 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 01:20:55.130013 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:20:55.130013 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:55.130013 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:55.130013 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:55.130013 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:55.130013 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:55.130013 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:55.130013 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:55.130013 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:55.130013 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:55.130013 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:55.130013 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:55.130013 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:55.130969 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:55.130969 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:55.130969 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 01:20:55.130969 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 01:20:55.130969 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 01:20:55.130969 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 01:20:55.130969 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 01:20:55.130969 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 01:20:55.130969 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 06 01:20:55.130969 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:20:55.135854 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 336/677] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Aug 6 01:20:54 2022] GET /identity/v3/users/swift => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:20:55.137360 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafba1ca-8df0-4ea3-846d-339bb54ba149 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:55.175597 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:55.221622 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43522018-e7ed-471f-9470-36c2465a7aa3 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:55.249634 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafba1ca-8df0-4ea3-846d-339bb54ba149 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:55.259044 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1a4838-b768-4f4e-bd41-f88c5d3330ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:55.260026 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1a4838-b768-4f4e-bd41-f88c5d3330ed None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:55.260319 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1a4838-b768-4f4e-bd41-f88c5d3330ed None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:55.260601 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1a4838-b768-4f4e-bd41-f88c5d3330ed None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:55.261732 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1a4838-b768-4f4e-bd41-f88c5d3330ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:55.262349 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1a4838-b768-4f4e-bd41-f88c5d3330ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:55.263723 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1a4838-b768-4f4e-bd41-f88c5d3330ed None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:55.277189 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 342/678] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Aug 6 01:20:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:55.277189 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1a4838-b768-4f4e-bd41-f88c5d3330ed None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:55.286730 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a09af9-b3d2-4588-adb4-abf17e11c04e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:55.287417 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a09af9-b3d2-4588-adb4-abf17e11c04e None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:55.287806 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a09af9-b3d2-4588-adb4-abf17e11c04e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:55.288089 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a09af9-b3d2-4588-adb4-abf17e11c04e None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:55.288849 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a09af9-b3d2-4588-adb4-abf17e11c04e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:55.289188 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a09af9-b3d2-4588-adb4-abf17e11c04e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:55.290733 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a09af9-b3d2-4588-adb4-abf17e11c04e None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:55.297464 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:55.311162 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 337/679] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Aug 6 01:20:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:55.311162 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a16ea57-d30e-4e01-8b05-433dd11cde87 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:55.311162 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a16ea57-d30e-4e01-8b05-433dd11cde87 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:55.311162 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a16ea57-d30e-4e01-8b05-433dd11cde87 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:55.311162 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 338/680] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:20:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:20:55.311162 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a16ea57-d30e-4e01-8b05-433dd11cde87 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:55.391757 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1a4838-b768-4f4e-bd41-f88c5d3330ed None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:55.428148 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:55.432278 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275fde48-a2c2-4800-82bf-eddefe7ad346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:55.433564 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275fde48-a2c2-4800-82bf-eddefe7ad346 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:55.434086 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275fde48-a2c2-4800-82bf-eddefe7ad346 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:55.434422 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275fde48-a2c2-4800-82bf-eddefe7ad346 None admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:55.435187 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275fde48-a2c2-4800-82bf-eddefe7ad346 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:55.435495 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275fde48-a2c2-4800-82bf-eddefe7ad346 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:55.435920 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a16ea57-d30e-4e01-8b05-433dd11cde87 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:55.436942 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275fde48-a2c2-4800-82bf-eddefe7ad346 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:55.459362 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 343/681] 10.222.0.26 () {56 vars in 1135 bytes} [Sat Aug 6 01:20:55 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:55.459362 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275fde48-a2c2-4800-82bf-eddefe7ad346 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:55.474465 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7206a33a-a1ea-4143-bd50-58e20da550c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:55.475320 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7206a33a-a1ea-4143-bd50-58e20da550c3 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:55.475615 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7206a33a-a1ea-4143-bd50-58e20da550c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:55.476149 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7206a33a-a1ea-4143-bd50-58e20da550c3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:55.483568 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7206a33a-a1ea-4143-bd50-58e20da550c3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:55.485024 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7206a33a-a1ea-4143-bd50-58e20da550c3 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:55.497029 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-7206a33a-a1ea-4143-bd50-58e20da550c3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:20:55.497029 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:55.497029 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:55.497029 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:55.497029 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:55.497029 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:55.497029 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:55.497029 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:55.497029 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:55.497029 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:55.497029 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:55.497029 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:55.497029 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 01:20:55.497029 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 01:20:55.497029 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 01:20:55.498021 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 01:20:55.498021 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:55.498021 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:55.498021 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:55.498021 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:55.498021 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:55.498021 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:55.498021 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:55.498021 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:55.498021 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:55.498021 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:55.498021 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:55.498021 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:55.498021 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:55.498021 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:55.498021 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:55.498021 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:55.498929 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 01:20:55.498929 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 01:20:55.498929 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 01:20:55.498929 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 01:20:55.498929 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 01:20:55.498929 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 01:20:55.498929 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:20:55.498929 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:20:55.498929 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 339/682] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Aug 6 01:20:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:20:55.499408 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55136fc-ceeb-4ebe-a2b1-35f16a61f195 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:55.499934 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55136fc-ceeb-4ebe-a2b1-35f16a61f195 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:55.499934 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55136fc-ceeb-4ebe-a2b1-35f16a61f195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:55.517320 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-d55136fc-ceeb-4ebe-a2b1-35f16a61f195 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:20:55.518522 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-d55136fc-ceeb-4ebe-a2b1-35f16a61f195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:20:55.549440 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:55.573948 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275fde48-a2c2-4800-82bf-eddefe7ad346 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:55.593095 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 340/683] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:20:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:20:55.593095 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55136fc-ceeb-4ebe-a2b1-35f16a61f195 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:55.622040 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c9d447-7d18-44ac-adea-12b0347024f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:55.622831 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c9d447-7d18-44ac-adea-12b0347024f5 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:55.623187 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c9d447-7d18-44ac-adea-12b0347024f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:55.623537 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c9d447-7d18-44ac-adea-12b0347024f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:55.624399 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c9d447-7d18-44ac-adea-12b0347024f5 None admin] RBAC: Adding query filter params (role.id=c9ce798acd4b482f8f391c6af97eb7a4, scope.project.id=d9d68d310bf24449a5fe20d2dc264a58, user.id=1be1e6173e374a60b9e1fbf51a1ab571) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:55.624781 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c9d447-7d18-44ac-adea-12b0347024f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:55.626410 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c9d447-7d18-44ac-adea-12b0347024f5 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:55.632594 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 344/684] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Aug 6 01:20:55 2022] GET /identity/v3/role_assignments?user.id=1be1e6173e374a60b9e1fbf51a1ab571&scope.project.id=d9d68d310bf24449a5fe20d2dc264a58&role.id=c9ce798acd4b482f8f391c6af97eb7a4 => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:55.633481 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c9d447-7d18-44ac-adea-12b0347024f5 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:55.726828 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:55.751431 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55136fc-ceeb-4ebe-a2b1-35f16a61f195 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:55.773348 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c9d447-7d18-44ac-adea-12b0347024f5 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:55.794347 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255b9502-941d-4c89-a3d1-f70476c70961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:55.795282 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255b9502-941d-4c89-a3d1-f70476c70961 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:55.795716 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255b9502-941d-4c89-a3d1-f70476c70961 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:55.796235 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255b9502-941d-4c89-a3d1-f70476c70961 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:55.821639 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9f25ee-5f2e-4311-b9b0-6a8a2bc8c297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:55.822329 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9f25ee-5f2e-4311-b9b0-6a8a2bc8c297 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:55.822619 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9f25ee-5f2e-4311-b9b0-6a8a2bc8c297 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:55.822891 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9f25ee-5f2e-4311-b9b0-6a8a2bc8c297 None admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:55.823931 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9f25ee-5f2e-4311-b9b0-6a8a2bc8c297 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:55.824249 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9f25ee-5f2e-4311-b9b0-6a8a2bc8c297 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:55.825800 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9f25ee-5f2e-4311-b9b0-6a8a2bc8c297 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:55.837253 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:55.841860 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 345/685] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Aug 6 01:20:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:55.842781 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9f25ee-5f2e-4311-b9b0-6a8a2bc8c297 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:55.878685 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255b9502-941d-4c89-a3d1-f70476c70961 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:55.880386 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255b9502-941d-4c89-a3d1-f70476c70961 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:55.969732 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:55.972150 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9f25ee-5f2e-4311-b9b0-6a8a2bc8c297 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:55.992312 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 341/686] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Aug 6 01:20:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 01:20:55.993318 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255b9502-941d-4c89-a3d1-f70476c70961 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:56.016639 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84a242f-1292-44a1-8fca-ba78a1a4770c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:56.017313 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84a242f-1292-44a1-8fca-ba78a1a4770c None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:56.017753 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84a242f-1292-44a1-8fca-ba78a1a4770c None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:56.018266 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84a242f-1292-44a1-8fca-ba78a1a4770c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:56.032616 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84a242f-1292-44a1-8fca-ba78a1a4770c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:56.034244 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84a242f-1292-44a1-8fca-ba78a1a4770c None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:56.050799 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-d84a242f-1292-44a1-8fca-ba78a1a4770c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:20:56.050799 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:56.050799 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:56.050799 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:56.050799 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:56.050799 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:56.050799 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:56.050799 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:56.050799 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:56.050799 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:56.050799 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:56.050799 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:56.050799 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:20:56.050799 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:20:56.050799 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:20:56.051926 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:20:56.051926 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:56.051926 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:56.051926 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:56.051926 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:56.051926 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:56.051926 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:56.051926 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:56.051926 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:56.051926 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:56.051926 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:56.051926 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:56.051926 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:56.051926 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:56.051926 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:56.051926 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:56.051926 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:56.053017 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:20:56.053017 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:20:56.053017 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:20:56.053017 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:20:56.057343 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 346/687] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:20:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:20:56.058305 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84a242f-1292-44a1-8fca-ba78a1a4770c None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:56.111526 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255b9502-941d-4c89-a3d1-f70476c70961 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:56.114783 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:56.156693 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b11aa30-c629-4d77-aa45-10332f77bb4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:56.157362 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b11aa30-c629-4d77-aa45-10332f77bb4b None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:56.157644 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b11aa30-c629-4d77-aa45-10332f77bb4b None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:56.157919 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b11aa30-c629-4d77-aa45-10332f77bb4b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:56.158875 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b11aa30-c629-4d77-aa45-10332f77bb4b None admin] RBAC: Adding query filter params (role.id=c9ce798acd4b482f8f391c6af97eb7a4, scope.project.id=d9d68d310bf24449a5fe20d2dc264a58, user.id=4ac0e31c84b4445fa167bbe4954066c2) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:56.159494 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b11aa30-c629-4d77-aa45-10332f77bb4b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:56.161224 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b11aa30-c629-4d77-aa45-10332f77bb4b None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:56.168589 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84a242f-1292-44a1-8fca-ba78a1a4770c None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:56.173140 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 342/688] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Aug 6 01:20:55 2022] GET /identity/v3/role_assignments?user.id=4ac0e31c84b4445fa167bbe4954066c2&scope.project.id=d9d68d310bf24449a5fe20d2dc264a58&role.id=c9ce798acd4b482f8f391c6af97eb7a4 => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:56.174238 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b11aa30-c629-4d77-aa45-10332f77bb4b None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:56.236057 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80a4f74-59fd-4db2-8501-7ce9e06a81b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:56.236724 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80a4f74-59fd-4db2-8501-7ce9e06a81b4 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:56.237007 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80a4f74-59fd-4db2-8501-7ce9e06a81b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:56.238509 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80a4f74-59fd-4db2-8501-7ce9e06a81b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:56.264837 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80a4f74-59fd-4db2-8501-7ce9e06a81b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:56.266278 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80a4f74-59fd-4db2-8501-7ce9e06a81b4 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:56.283677 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:56.301522 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-d80a4f74-59fd-4db2-8501-7ce9e06a81b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:20:56.301522 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:56.301522 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:56.301522 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:56.301522 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:56.301522 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:56.301522 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:56.301522 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:56.301522 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:56.301522 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:56.301522 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:56.301522 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:56.301522 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:20:56.301522 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:20:56.301522 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:20:56.302493 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:20:56.302493 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:56.302493 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:56.302493 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:56.302493 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:56.302493 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:56.302493 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:56.302493 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:56.302493 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:56.302493 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:56.302493 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:56.302493 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:56.302493 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:56.302493 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:56.302493 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:56.302493 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:56.302493 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:56.303377 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:20:56.303377 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:20:56.303377 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:20:56.303377 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:20:56.303377 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 347/689] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:20:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:20:56.309691 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80a4f74-59fd-4db2-8501-7ce9e06a81b4 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:56.375063 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b11aa30-c629-4d77-aa45-10332f77bb4b None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:56.429485 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d824c6-6b0f-466c-9a37-32ebfe6bad2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:56.430550 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d824c6-6b0f-466c-9a37-32ebfe6bad2d None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:56.431049 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d824c6-6b0f-466c-9a37-32ebfe6bad2d None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:56.431610 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d824c6-6b0f-466c-9a37-32ebfe6bad2d None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:56.432507 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d824c6-6b0f-466c-9a37-32ebfe6bad2d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:56.433857 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d824c6-6b0f-466c-9a37-32ebfe6bad2d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:56.435637 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d824c6-6b0f-466c-9a37-32ebfe6bad2d None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:56.448859 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 343/690] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Aug 6 01:20:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:56.451823 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80a4f74-59fd-4db2-8501-7ce9e06a81b4 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:56.457289 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d824c6-6b0f-466c-9a37-32ebfe6bad2d None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:56.461118 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:56.507361 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff71e8d8-c2a1-4351-bcbc-b6abc9e69a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:56.508166 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff71e8d8-c2a1-4351-bcbc-b6abc9e69a5f None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:56.508486 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff71e8d8-c2a1-4351-bcbc-b6abc9e69a5f None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:56.508767 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff71e8d8-c2a1-4351-bcbc-b6abc9e69a5f None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:56.509523 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff71e8d8-c2a1-4351-bcbc-b6abc9e69a5f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:56.509831 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff71e8d8-c2a1-4351-bcbc-b6abc9e69a5f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:56.511301 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff71e8d8-c2a1-4351-bcbc-b6abc9e69a5f None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:56.527550 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 348/691] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Aug 6 01:20:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:56.532866 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff71e8d8-c2a1-4351-bcbc-b6abc9e69a5f None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:56.580557 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d824c6-6b0f-466c-9a37-32ebfe6bad2d None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:56.593042 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:56.617995 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d631cf-6366-4704-b925-d61589a9557f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:56.619071 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d631cf-6366-4704-b925-d61589a9557f None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:56.619552 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d631cf-6366-4704-b925-d61589a9557f None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:56.620046 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d631cf-6366-4704-b925-d61589a9557f None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:56.621502 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d631cf-6366-4704-b925-d61589a9557f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:56.623260 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d631cf-6366-4704-b925-d61589a9557f None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:56.643634 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff71e8d8-c2a1-4351-bcbc-b6abc9e69a5f None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:56.648273 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 344/692] 10.222.0.26 () {60 vars in 1132 bytes} [Sat Aug 6 01:20:56 2022] POST /identity/v3/users => generated 254 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:20:56.650615 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb9ef4e-5186-4b27-880c-55f2d6135b3c None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:56.650906 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb9ef4e-5186-4b27-880c-55f2d6135b3c None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:56.651182 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb9ef4e-5186-4b27-880c-55f2d6135b3c None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:56.652897 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 345/693] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:20:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 01:20:56.658353 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36152cfe-c9a3-4955-8e34-d6fbc10d9509 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:56.658635 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36152cfe-c9a3-4955-8e34-d6fbc10d9509 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:56.658920 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36152cfe-c9a3-4955-8e34-d6fbc10d9509 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:56.678606 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-36152cfe-c9a3-4955-8e34-d6fbc10d9509 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:20:56.679994 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-36152cfe-c9a3-4955-8e34-d6fbc10d9509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:20:56.709579 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c58d5d-4bd6-4770-b3ab-3bf2b02b8d1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:56.710250 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c58d5d-4bd6-4770-b3ab-3bf2b02b8d1f None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:56.710537 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c58d5d-4bd6-4770-b3ab-3bf2b02b8d1f None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:56.710814 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c58d5d-4bd6-4770-b3ab-3bf2b02b8d1f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:56.724224 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c58d5d-4bd6-4770-b3ab-3bf2b02b8d1f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:56.725741 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c58d5d-4bd6-4770-b3ab-3bf2b02b8d1f None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:56.753429 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-57c58d5d-4bd6-4770-b3ab-3bf2b02b8d1f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:20:56.753429 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:56.753429 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:56.753429 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:56.753429 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:56.753429 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:56.753429 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:56.753429 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:56.753429 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:56.753429 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:56.753429 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:56.753429 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:56.753429 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 01:20:56.753429 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 01:20:56.753429 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 01:20:56.754360 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 01:20:56.754360 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:56.754360 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:56.754360 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:56.754360 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:56.754360 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:56.754360 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:56.754360 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:56.754360 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:56.754360 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:56.754360 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:56.754360 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:56.754360 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:56.754360 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:56.754360 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:56.754360 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:56.754360 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:56.755276 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 01:20:56.755276 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 01:20:56.755276 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 01:20:56.755276 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 01:20:56.755276 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 01:20:56.755276 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 01:20:56.755276 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:20:56.755276 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:20:56.755276 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 349/695] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Aug 6 01:20:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:20:56.755753 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 346/695] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:20:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 01:20:56.762999 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c58d5d-4bd6-4770-b3ab-3bf2b02b8d1f None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:56.797659 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:56.804328 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36152cfe-c9a3-4955-8e34-d6fbc10d9509 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:56.945268 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c58d5d-4bd6-4770-b3ab-3bf2b02b8d1f None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:56.958023 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:56.967348 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36152cfe-c9a3-4955-8e34-d6fbc10d9509 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:56.987258 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c682c12a-cf95-453d-814f-89f66b361d9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:56.988259 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c682c12a-cf95-453d-814f-89f66b361d9d None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:56.988714 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c682c12a-cf95-453d-814f-89f66b361d9d None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:56.989194 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c682c12a-cf95-453d-814f-89f66b361d9d None admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:56.990133 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c682c12a-cf95-453d-814f-89f66b361d9d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:56.990728 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c682c12a-cf95-453d-814f-89f66b361d9d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:56.992355 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c682c12a-cf95-453d-814f-89f66b361d9d None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:57.007071 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 350/696] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Aug 6 01:20:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:57.012791 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b8c229-d4c1-4e65-904a-edf2bd2b7691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:57.012791 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b8c229-d4c1-4e65-904a-edf2bd2b7691 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:57.012791 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b8c229-d4c1-4e65-904a-edf2bd2b7691 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:57.012791 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b8c229-d4c1-4e65-904a-edf2bd2b7691 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:57.014383 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c682c12a-cf95-453d-814f-89f66b361d9d None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:57.025873 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b8c229-d4c1-4e65-904a-edf2bd2b7691 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:20:57.032089 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-88b8c229-d4c1-4e65-904a-edf2bd2b7691 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:20:57.032089 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:57.032089 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:57.032089 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:57.032089 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:57.032089 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:57.032089 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:57.032089 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:57.032089 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:57.032089 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:57.032089 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:57.032089 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:57.032089 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:20:57.032089 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:20:57.032089 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:20:57.032089 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:20:57.033239 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:20:57.033239 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:20:57.033239 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:20:57.033239 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:20:57.033239 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:57.033239 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:57.033239 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:57.033239 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:57.033239 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:57.033239 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:57.033239 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:57.033239 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:57.033239 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:57.033239 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:57.033239 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:57.033239 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:57.034265 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:57.034265 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:57.034265 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:57.034265 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:57.034265 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:20:57.034265 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:20:57.034265 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:20:57.034265 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:20:57.034265 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:20:57.034265 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:20:57.034265 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:20:57.034265 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:20:57.036260 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 347/697] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Aug 6 01:20:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:20:57.043847 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b8c229-d4c1-4e65-904a-edf2bd2b7691 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:57.124840 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:57.154849 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c682c12a-cf95-453d-814f-89f66b361d9d None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:57.201422 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b8c229-d4c1-4e65-904a-edf2bd2b7691 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:57.215224 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2482741e-e5e2-48a2-9438-7ac227ed5028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:57.215853 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2482741e-e5e2-48a2-9438-7ac227ed5028 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:57.216129 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2482741e-e5e2-48a2-9438-7ac227ed5028 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:57.216451 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2482741e-e5e2-48a2-9438-7ac227ed5028 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:57.217274 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2482741e-e5e2-48a2-9438-7ac227ed5028 None admin] RBAC: Adding query filter params (role.id=c9ce798acd4b482f8f391c6af97eb7a4, scope.project.id=d9d68d310bf24449a5fe20d2dc264a58, user.id=2b23f4478f49471fba6f43e6e4be99da) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:57.217584 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2482741e-e5e2-48a2-9438-7ac227ed5028 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:57.219015 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2482741e-e5e2-48a2-9438-7ac227ed5028 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:57.233885 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 351/698] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Aug 6 01:20:57 2022] GET /identity/v3/role_assignments?user.id=2b23f4478f49471fba6f43e6e4be99da&scope.project.id=d9d68d310bf24449a5fe20d2dc264a58&role.id=c9ce798acd4b482f8f391c6af97eb7a4 => generated 608 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:57.263289 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d524e4f6-5bae-4799-bc2e-655a5ba6aa17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:57.263919 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d524e4f6-5bae-4799-bc2e-655a5ba6aa17 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:57.264206 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d524e4f6-5bae-4799-bc2e-655a5ba6aa17 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:57.264484 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d524e4f6-5bae-4799-bc2e-655a5ba6aa17 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:57.265231 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d524e4f6-5bae-4799-bc2e-655a5ba6aa17 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:57.265540 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d524e4f6-5bae-4799-bc2e-655a5ba6aa17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:57.270546 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d524e4f6-5bae-4799-bc2e-655a5ba6aa17 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:57.280403 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 348/699] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Aug 6 01:20:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:20:57.286223 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2482741e-e5e2-48a2-9438-7ac227ed5028 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:57.459297 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2482741e-e5e2-48a2-9438-7ac227ed5028 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:57.509423 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583e2f05-c798-4060-83e8-f3da21b8e285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:57.510449 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583e2f05-c798-4060-83e8-f3da21b8e285 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:57.511647 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583e2f05-c798-4060-83e8-f3da21b8e285 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:57.512267 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583e2f05-c798-4060-83e8-f3da21b8e285 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:57.524156 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583e2f05-c798-4060-83e8-f3da21b8e285 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:57.525913 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583e2f05-c798-4060-83e8-f3da21b8e285 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:57.534735 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:57.542067 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-583e2f05-c798-4060-83e8-f3da21b8e285 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:20:57.542067 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:57.542067 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:57.542067 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:57.542067 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:57.542067 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:57.542067 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:57.542067 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:57.542067 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:57.542067 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:57.542067 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:57.542067 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:57.542067 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:20:57.542067 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:20:57.542067 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:20:57.542984 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:20:57.542984 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:57.542984 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:57.542984 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:57.542984 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:57.542984 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:57.542984 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:57.542984 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:57.542984 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:57.542984 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:57.542984 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:57.542984 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:57.542984 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:57.542984 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:57.542984 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:57.542984 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:57.542984 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:57.543890 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:20:57.543890 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:20:57.543890 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:20:57.543890 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:20:57.546108 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 352/700] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:20:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:20:57.551906 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d524e4f6-5bae-4799-bc2e-655a5ba6aa17 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:57.575039 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:57.662792 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d524e4f6-5bae-4799-bc2e-655a5ba6aa17 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:57.700359 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be07a663-efd6-45a3-b794-a502cace2145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:57.701409 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be07a663-efd6-45a3-b794-a502cace2145 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:57.701966 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be07a663-efd6-45a3-b794-a502cace2145 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:57.702476 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be07a663-efd6-45a3-b794-a502cace2145 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:57.703511 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be07a663-efd6-45a3-b794-a502cace2145 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:57.704034 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be07a663-efd6-45a3-b794-a502cace2145 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:57.705684 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be07a663-efd6-45a3-b794-a502cace2145 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:57.719711 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 349/701] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Aug 6 01:20:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:57.725464 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583e2f05-c798-4060-83e8-f3da21b8e285 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:57.832744 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583e2f05-c798-4060-83e8-f3da21b8e285 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:57.868707 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab428e2-b581-4f23-baff-785fc31b72b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:57.869756 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab428e2-b581-4f23-baff-785fc31b72b8 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:57.870273 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab428e2-b581-4f23-baff-785fc31b72b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:57.870773 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab428e2-b581-4f23-baff-785fc31b72b8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:57.878781 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab428e2-b581-4f23-baff-785fc31b72b8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:57.880452 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab428e2-b581-4f23-baff-785fc31b72b8 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:57.892775 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-8ab428e2-b581-4f23-baff-785fc31b72b8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 06 01:20:57.892775 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:57.892775 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:57.892775 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:57.892775 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:57.892775 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:57.892775 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:57.892775 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:57.892775 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:57.892775 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:57.892775 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:57.892775 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:57.892775 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 01:20:57.892775 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 01:20:57.892775 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 01:20:57.892775 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 01:20:57.893766 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:57.893766 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:57.893766 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 01:20:57.893766 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:20:57.893766 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 01:20:57.893766 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:20:57.893766 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:57.893766 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:57.893766 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:57.893766 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:57.893766 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:57.893766 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:57.893766 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:57.893766 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:57.893766 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:57.893766 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:57.893766 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:57.893766 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:57.894738 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:57.894738 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:57.894738 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 01:20:57.894738 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 01:20:57.894738 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 01:20:57.894738 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 01:20:57.894738 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 01:20:57.894738 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 01:20:57.894738 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 06 01:20:57.894738 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:20:57.901383 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 353/702] 10.222.0.26 () {56 vars in 1093 bytes} [Sat Aug 6 01:20:57 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:20:57.903930 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be07a663-efd6-45a3-b794-a502cace2145 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:57.985688 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:58.009294 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be07a663-efd6-45a3-b794-a502cace2145 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:58.046041 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaa54fc-aa48-4b5c-a1eb-924f83d1e613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:58.046668 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaa54fc-aa48-4b5c-a1eb-924f83d1e613 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:58.046948 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaa54fc-aa48-4b5c-a1eb-924f83d1e613 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:58.047223 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaa54fc-aa48-4b5c-a1eb-924f83d1e613 None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:58.048015 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaa54fc-aa48-4b5c-a1eb-924f83d1e613 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:58.048324 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaa54fc-aa48-4b5c-a1eb-924f83d1e613 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:58.049784 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaa54fc-aa48-4b5c-a1eb-924f83d1e613 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:58.062680 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cd5832-70c4-4264-9558-f9103b51ea15 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:58.062965 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cd5832-70c4-4264-9558-f9103b51ea15 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:58.063308 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cd5832-70c4-4264-9558-f9103b51ea15 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:58.064791 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 354/703] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:20:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 01:20:58.070442 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d5a0b4-fc8e-4413-b5a7-8d40c2f6b92a None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:58.071091 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d5a0b4-fc8e-4413-b5a7-8d40c2f6b92a None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:58.071646 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d5a0b4-fc8e-4413-b5a7-8d40c2f6b92a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:58.072503 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 350/704] 10.222.0.26 () {56 vars in 1133 bytes} [Sat Aug 6 01:20:57 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:58.078526 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaa54fc-aa48-4b5c-a1eb-924f83d1e613 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:58.094388 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-85d5a0b4-fc8e-4413-b5a7-8d40c2f6b92a None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:20:58.095916 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-85d5a0b4-fc8e-4413-b5a7-8d40c2f6b92a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:20:58.199633 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:58.224645 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 355/705] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:20:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 01:20:58.250819 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaa54fc-aa48-4b5c-a1eb-924f83d1e613 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:58.269759 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d5a0b4-fc8e-4413-b5a7-8d40c2f6b92a None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:58.300799 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0776fc05-4749-47ca-b5fd-eed6cc72744e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:58.301957 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0776fc05-4749-47ca-b5fd-eed6cc72744e None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:58.302632 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0776fc05-4749-47ca-b5fd-eed6cc72744e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:58.303865 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0776fc05-4749-47ca-b5fd-eed6cc72744e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:58.312683 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:58.428560 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d5a0b4-fc8e-4413-b5a7-8d40c2f6b92a None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:58.464439 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0776fc05-4749-47ca-b5fd-eed6cc72744e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:58.466837 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0776fc05-4749-47ca-b5fd-eed6cc72744e None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:58.493164 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0c1cee-d1ce-4a34-8e68-5256dfad0f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:58.493915 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0c1cee-d1ce-4a34-8e68-5256dfad0f79 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:58.494089 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0c1cee-d1ce-4a34-8e68-5256dfad0f79 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:58.494365 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0c1cee-d1ce-4a34-8e68-5256dfad0f79 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:58.494996 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0c1cee-d1ce-4a34-8e68-5256dfad0f79 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:58.505591 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0c1cee-d1ce-4a34-8e68-5256dfad0f79 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:58.528423 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-df0c1cee-d1ce-4a34-8e68-5256dfad0f79 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 06 01:20:58.528423 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:58.528423 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 01:20:58.528423 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 01:20:58.528423 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 01:20:58.528423 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 01:20:58.528423 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 01:20:58.528423 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 01:20:58.528423 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 06 01:20:58.528423 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:20:58.528423 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 01:20:58.528423 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:20:58.528423 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:58.528423 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:58.528423 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:58.528423 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:58.529553 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:58.529553 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:58.529553 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:58.529553 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:58.529553 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:58.529553 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:58.529553 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:58.529553 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 01:20:58.529553 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 01:20:58.529553 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 01:20:58.529553 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 01:20:58.529553 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:58.529553 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:58.529553 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:58.529553 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:58.529553 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:58.530410 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:58.530410 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:58.530410 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:58.530410 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:58.530410 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:58.530410 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:58.530410 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:58.530410 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:58.530410 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:58.530410 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:58.530410 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:58.530410 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 01:20:58.530410 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 01:20:58.530410 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 06 01:20:58.530410 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:20:58.530410 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 356/706] 10.222.0.26 () {56 vars in 1129 bytes} [Sat Aug 6 01:20:58 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:20:58.542685 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0c1cee-d1ce-4a34-8e68-5256dfad0f79 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:58.590143 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 351/707] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Aug 6 01:20:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 512 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 01:20:58.595888 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0776fc05-4749-47ca-b5fd-eed6cc72744e None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:58.649405 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:58.661369 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:58.749956 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0c1cee-d1ce-4a34-8e68-5256dfad0f79 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:58.795096 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0776fc05-4749-47ca-b5fd-eed6cc72744e None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:58.810353 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a97d01c-03bf-443e-a14b-1448bb1ccfe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:58.810980 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a97d01c-03bf-443e-a14b-1448bb1ccfe4 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:58.811624 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a97d01c-03bf-443e-a14b-1448bb1ccfe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:58.811903 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a97d01c-03bf-443e-a14b-1448bb1ccfe4 None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:58.812879 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a97d01c-03bf-443e-a14b-1448bb1ccfe4 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:58.813231 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a97d01c-03bf-443e-a14b-1448bb1ccfe4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:58.814902 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a97d01c-03bf-443e-a14b-1448bb1ccfe4 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:58.825223 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 357/708] 10.222.0.26 () {56 vars in 1124 bytes} [Sat Aug 6 01:20:58 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:58.826554 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ec241a-ce2b-4d02-af8e-8e6c37b87d75 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:58.826943 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ec241a-ce2b-4d02-af8e-8e6c37b87d75 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:58.827183 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ec241a-ce2b-4d02-af8e-8e6c37b87d75 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:58.828566 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 358/709] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:20:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:20:58.830857 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ec241a-ce2b-4d02-af8e-8e6c37b87d75 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:58.847077 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d722303f-b74f-4b23-a85a-765131cbd433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:58.848323 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d722303f-b74f-4b23-a85a-765131cbd433 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:58.848805 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d722303f-b74f-4b23-a85a-765131cbd433 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:58.849538 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d722303f-b74f-4b23-a85a-765131cbd433 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:58.863746 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d722303f-b74f-4b23-a85a-765131cbd433 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:58.866078 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d722303f-b74f-4b23-a85a-765131cbd433 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:58.887162 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-d722303f-b74f-4b23-a85a-765131cbd433 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:20:58.887162 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:58.887162 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:58.887162 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:58.887162 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:58.887162 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:58.887162 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:58.887162 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:58.887162 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:58.887162 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:58.887162 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:58.887162 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:58.887162 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:20:58.887162 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:20:58.887162 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:20:58.888274 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:20:58.888274 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:58.888274 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:58.888274 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:58.888274 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:58.888274 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:58.888274 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:58.888274 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:58.888274 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:58.888274 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:58.888274 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:58.888274 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:58.888274 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:58.888274 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:58.888274 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:58.888274 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:58.888274 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:58.889348 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:20:58.889348 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:20:58.889348 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:20:58.889348 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:20:58.894910 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 352/710] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:20:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:20:58.894910 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781ea2b4-ec44-499e-ac06-9c7b47d58ff7 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:58.894910 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781ea2b4-ec44-499e-ac06-9c7b47d58ff7 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:58.894910 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781ea2b4-ec44-499e-ac06-9c7b47d58ff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:58.917010 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-781ea2b4-ec44-499e-ac06-9c7b47d58ff7 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:20:58.918851 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-781ea2b4-ec44-499e-ac06-9c7b47d58ff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:20:58.962520 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ec241a-ce2b-4d02-af8e-8e6c37b87d75 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:58.986352 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:59.013831 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df867df2-0b45-4f04-8de4-9db02f4bdc0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:59.013831 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df867df2-0b45-4f04-8de4-9db02f4bdc0a None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:59.013831 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df867df2-0b45-4f04-8de4-9db02f4bdc0a None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:59.013831 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df867df2-0b45-4f04-8de4-9db02f4bdc0a None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:59.014231 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df867df2-0b45-4f04-8de4-9db02f4bdc0a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:59.014231 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df867df2-0b45-4f04-8de4-9db02f4bdc0a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:59.014231 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df867df2-0b45-4f04-8de4-9db02f4bdc0a None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:59.014946 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 353/711] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:20:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:20:59.016279 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781ea2b4-ec44-499e-ac06-9c7b47d58ff7 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:59.027456 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 359/712] 10.222.0.26 () {56 vars in 1124 bytes} [Sat Aug 6 01:20:58 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:59.048349 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:59.071865 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df867df2-0b45-4f04-8de4-9db02f4bdc0a None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:59.207949 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781ea2b4-ec44-499e-ac06-9c7b47d58ff7 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:59.222160 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df867df2-0b45-4f04-8de4-9db02f4bdc0a None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:59.256482 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a29f2b-8f7e-4616-80bb-fd99304176f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:59.257526 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a29f2b-8f7e-4616-80bb-fd99304176f7 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:59.257813 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a29f2b-8f7e-4616-80bb-fd99304176f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:59.258359 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a29f2b-8f7e-4616-80bb-fd99304176f7 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:59.259158 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a29f2b-8f7e-4616-80bb-fd99304176f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:59.259469 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a29f2b-8f7e-4616-80bb-fd99304176f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:59.262066 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a29f2b-8f7e-4616-80bb-fd99304176f7 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:59.263445 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f915a70-a160-44f8-ba5d-0b067ffe157d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:59.263972 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f915a70-a160-44f8-ba5d-0b067ffe157d None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:59.265082 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f915a70-a160-44f8-ba5d-0b067ffe157d None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:59.265082 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f915a70-a160-44f8-ba5d-0b067ffe157d None admin] PATH_INFO: `/v3/services/network` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:59.265248 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f915a70-a160-44f8-ba5d-0b067ffe157d None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:59.267472 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f915a70-a160-44f8-ba5d-0b067ffe157d None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:59.275908 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 354/713] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Aug 6 01:20:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:59.277301 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51527d9-e051-4e02-aeac-ffe6af2bc69d None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:59.277586 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51527d9-e051-4e02-aeac-ffe6af2bc69d None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:59.277862 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51527d9-e051-4e02-aeac-ffe6af2bc69d None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:59.278794 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-4f915a70-a160-44f8-ba5d-0b067ffe157d None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 06 01:20:59.278794 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:59.278794 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 01:20:59.278794 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 01:20:59.278794 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 01:20:59.278794 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 01:20:59.278794 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 01:20:59.278794 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 01:20:59.278794 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 06 01:20:59.278794 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:20:59.278794 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 01:20:59.278794 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:20:59.278794 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:59.278794 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:59.278794 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:59.278794 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:59.278794 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:59.279823 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:59.279823 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:59.279823 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:59.279823 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:59.279823 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:59.279823 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:59.279823 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 01:20:59.279823 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 01:20:59.279823 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 01:20:59.279823 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 01:20:59.279823 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:59.279823 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:59.279823 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:59.279823 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:59.279823 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:59.279823 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:59.280664 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:59.280664 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:59.280664 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:59.280664 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:59.280664 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:59.280664 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:59.280664 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:59.280664 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:59.280664 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:59.280664 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:59.280664 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 01:20:59.280664 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 01:20:59.280664 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 06 01:20:59.280664 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:20:59.281445 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 355/714] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:20:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:20:59.283743 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 360/715] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Aug 6 01:20:59 2022] GET /identity/v3/services/network => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:20:59.286468 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfb0dcb-6599-4c65-8d06-a9a121ade71d None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:59.286764 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfb0dcb-6599-4c65-8d06-a9a121ade71d None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:59.287323 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfb0dcb-6599-4c65-8d06-a9a121ade71d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:59.288125 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f915a70-a160-44f8-ba5d-0b067ffe157d None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:59.305844 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-adfb0dcb-6599-4c65-8d06-a9a121ade71d None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:20:59.307062 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-adfb0dcb-6599-4c65-8d06-a9a121ade71d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:20:59.357427 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:59.393098 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 356/716] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:20:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:20:59.393098 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfb0dcb-6599-4c65-8d06-a9a121ade71d None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:59.410103 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:59.432043 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f915a70-a160-44f8-ba5d-0b067ffe157d None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:59.469468 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b98876-379c-46c9-9142-ed0637070ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:59.470111 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b98876-379c-46c9-9142-ed0637070ebb None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:59.470395 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b98876-379c-46c9-9142-ed0637070ebb None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:59.470670 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b98876-379c-46c9-9142-ed0637070ebb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:59.478561 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b98876-379c-46c9-9142-ed0637070ebb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:59.480657 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b98876-379c-46c9-9142-ed0637070ebb None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:59.492902 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-c3b98876-379c-46c9-9142-ed0637070ebb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:20:59.492902 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:59.492902 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:59.492902 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:59.492902 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:59.492902 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:59.492902 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:59.492902 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:59.492902 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:59.492902 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:59.492902 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:59.492902 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:59.492902 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 01:20:59.492902 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 01:20:59.492902 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 01:20:59.493847 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 01:20:59.493847 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:59.493847 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:59.493847 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:59.493847 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:59.493847 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:59.493847 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:59.493847 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:59.493847 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:59.493847 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:59.493847 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:59.493847 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:59.493847 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:59.493847 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:59.493847 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:59.493847 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:59.493847 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:59.494773 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 01:20:59.494773 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 01:20:59.494773 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 01:20:59.494773 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 01:20:59.494773 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 01:20:59.494773 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 01:20:59.494773 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:20:59.494773 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:20:59.494773 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 361/717] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Aug 6 01:20:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:20:59.495847 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b98876-379c-46c9-9142-ed0637070ebb None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:59.529558 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfb0dcb-6599-4c65-8d06-a9a121ade71d None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:59.567266 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b4f5d1-6a7f-4a91-b406-1dc44e7177b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:59.568253 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b4f5d1-6a7f-4a91-b406-1dc44e7177b3 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:59.568749 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b4f5d1-6a7f-4a91-b406-1dc44e7177b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:59.569294 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b4f5d1-6a7f-4a91-b406-1dc44e7177b3 None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:59.570295 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b4f5d1-6a7f-4a91-b406-1dc44e7177b3 None admin] RBAC: Adding query filter params (name=network) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:59.570950 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b4f5d1-6a7f-4a91-b406-1dc44e7177b3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:59.572571 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b4f5d1-6a7f-4a91-b406-1dc44e7177b3 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:59.578233 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 357/718] 10.222.0.26 () {56 vars in 1112 bytes} [Sat Aug 6 01:20:59 2022] GET /identity/v3/services?name=network => generated 124 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:59.579366 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b4f5d1-6a7f-4a91-b406-1dc44e7177b3 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:59.618833 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b98876-379c-46c9-9142-ed0637070ebb None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:59.628073 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:59.661744 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:59.666646 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8568fc-87b4-45b7-9617-d4e93b7960bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:59.667630 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8568fc-87b4-45b7-9617-d4e93b7960bc None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:59.668946 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8568fc-87b4-45b7-9617-d4e93b7960bc None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:59.669464 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8568fc-87b4-45b7-9617-d4e93b7960bc None admin] PATH_INFO: `/v3/roles/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:59.684092 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8568fc-87b4-45b7-9617-d4e93b7960bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:20:59.689509 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-4f8568fc-87b4-45b7-9617-d4e93b7960bc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 06 01:20:59.689509 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:20:59.689509 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:20:59.689509 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:20:59.689509 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:20:59.689509 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:20:59.689509 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:20:59.689509 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:20:59.689509 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:20:59.689509 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:20:59.689509 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:20:59.689509 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:20:59.689509 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:20:59.689509 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:20:59.689509 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:20:59.689509 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:20:59.690433 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:20:59.690433 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:20:59.690433 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:20:59.690433 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:20:59.690433 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:20:59.690433 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:20:59.690433 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:20:59.690433 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:20:59.690433 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:20:59.690433 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:20:59.690433 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:20:59.690433 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:20:59.690433 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:20:59.690433 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:20:59.690433 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:20:59.690433 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:20:59.691263 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:20:59.691263 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:20:59.691263 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:20:59.691263 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:20:59.691263 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:20:59.691263 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:20:59.691263 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:20:59.691263 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:20:59.691263 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:20:59.691263 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:20:59.691263 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 06 01:20:59.691263 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:20:59.695575 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 362/719] 10.222.0.26 () {56 vars in 1102 bytes} [Sat Aug 6 01:20:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:20:59.696497 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8568fc-87b4-45b7-9617-d4e93b7960bc None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:59.702718 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b4f5d1-6a7f-4a91-b406-1dc44e7177b3 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:59.748348 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84091114-b832-48a3-a5fc-b8d7d8b78bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:59.749007 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84091114-b832-48a3-a5fc-b8d7d8b78bc8 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:59.749323 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84091114-b832-48a3-a5fc-b8d7d8b78bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:59.749604 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84091114-b832-48a3-a5fc-b8d7d8b78bc8 None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:59.750364 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84091114-b832-48a3-a5fc-b8d7d8b78bc8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:59.750673 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84091114-b832-48a3-a5fc-b8d7d8b78bc8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:59.752096 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84091114-b832-48a3-a5fc-b8d7d8b78bc8 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:59.773297 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 358/720] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Aug 6 01:20:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:59.773297 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84091114-b832-48a3-a5fc-b8d7d8b78bc8 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:59.842880 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8568fc-87b4-45b7-9617-d4e93b7960bc None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:59.893788 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ad6a57-ab02-4558-afc0-690dd4ea728a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:59.894982 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ad6a57-ab02-4558-afc0-690dd4ea728a None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:59.895698 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ad6a57-ab02-4558-afc0-690dd4ea728a None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:59.896170 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ad6a57-ab02-4558-afc0-690dd4ea728a None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:59.897663 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ad6a57-ab02-4558-afc0-690dd4ea728a None admin] RBAC: Adding query filter params (type=network) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:59.898179 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ad6a57-ab02-4558-afc0-690dd4ea728a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:59.899394 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84091114-b832-48a3-a5fc-b8d7d8b78bc8 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:59.900001 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ad6a57-ab02-4558-afc0-690dd4ea728a None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:59.906588 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 363/721] 10.222.0.26 () {56 vars in 1112 bytes} [Sat Aug 6 01:20:59 2022] GET /identity/v3/services?type=network => generated 124 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:59.907881 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ad6a57-ab02-4558-afc0-690dd4ea728a None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:59.950554 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:59.964841 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:59.975719 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6516e7c-46ee-4167-b45c-af34d0e79898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:59.978159 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6516e7c-46ee-4167-b45c-af34d0e79898 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:59.980353 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6516e7c-46ee-4167-b45c-af34d0e79898 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:59.980912 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6516e7c-46ee-4167-b45c-af34d0e79898 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:59.981910 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6516e7c-46ee-4167-b45c-af34d0e79898 None admin] RBAC: Adding query filter params (name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:20:59.983199 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6516e7c-46ee-4167-b45c-af34d0e79898 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:59.984829 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6516e7c-46ee-4167-b45c-af34d0e79898 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:00.009340 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 359/722] 10.222.0.26 () {56 vars in 1103 bytes} [Sat Aug 6 01:20:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:00.012404 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6516e7c-46ee-4167-b45c-af34d0e79898 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:00.089796 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ad6a57-ab02-4558-afc0-690dd4ea728a None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:00.129470 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e0af8a-9c6d-4378-9e07-cbd18b7a68b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:00.129827 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e0af8a-9c6d-4378-9e07-cbd18b7a68b5 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:00.130206 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e0af8a-9c6d-4378-9e07-cbd18b7a68b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:00.130544 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e0af8a-9c6d-4378-9e07-cbd18b7a68b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:00.131401 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e0af8a-9c6d-4378-9e07-cbd18b7a68b5 None admin] RBAC: Adding query filter params (role.id=4559b9a0837249f895a13c70096f8d38, scope.project.id=d9d68d310bf24449a5fe20d2dc264a58, user.id=b87adc1d5bc240e99f95f6595b350be4) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:00.131805 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e0af8a-9c6d-4378-9e07-cbd18b7a68b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:00.133339 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e0af8a-9c6d-4378-9e07-cbd18b7a68b5 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:00.140271 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 364/723] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Aug 6 01:20:59 2022] GET /identity/v3/role_assignments?user.id=b87adc1d5bc240e99f95f6595b350be4&scope.project.id=d9d68d310bf24449a5fe20d2dc264a58&role.id=4559b9a0837249f895a13c70096f8d38 => generated 259 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:00.148210 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6516e7c-46ee-4167-b45c-af34d0e79898 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:00.149468 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c31bc62-317f-4612-9515-f6e6780d0310 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:00.149792 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c31bc62-317f-4612-9515-f6e6780d0310 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:00.150073 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c31bc62-317f-4612-9515-f6e6780d0310 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:00.152484 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 365/724] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:21:00 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:21:00.159830 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3485cc-a764-4036-a938-a3c00fbdb5e3 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:00.161555 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3485cc-a764-4036-a938-a3c00fbdb5e3 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:00.161848 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3485cc-a764-4036-a938-a3c00fbdb5e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:00.200313 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-2d3485cc-a764-4036-a938-a3c00fbdb5e3 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:21:00.202947 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-2d3485cc-a764-4036-a938-a3c00fbdb5e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:21:00.209865 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe83a9a1-0161-4da2-a847-f63180434bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:00.210921 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe83a9a1-0161-4da2-a847-f63180434bcf None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:00.211411 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe83a9a1-0161-4da2-a847-f63180434bcf None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:00.211904 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe83a9a1-0161-4da2-a847-f63180434bcf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:00.234891 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe83a9a1-0161-4da2-a847-f63180434bcf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:00.236889 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe83a9a1-0161-4da2-a847-f63180434bcf None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:00.270169 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-fe83a9a1-0161-4da2-a847-f63180434bcf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:21:00.270169 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:00.270169 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:21:00.270169 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:21:00.270169 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:21:00.270169 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:21:00.270169 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:21:00.270169 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:21:00.270169 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:21:00.270169 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:21:00.270169 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:21:00.270169 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:21:00.270169 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:21:00.270169 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:21:00.270169 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:21:00.271596 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:21:00.271596 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:21:00.271596 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:21:00.271596 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:21:00.271596 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:21:00.271596 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:21:00.271596 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:21:00.271596 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:21:00.271596 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:21:00.271596 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:21:00.271596 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:21:00.271596 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:21:00.271596 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:21:00.271596 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:21:00.271596 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:21:00.271596 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:21:00.271596 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:21:00.272480 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:21:00.272480 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:21:00.272480 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:21:00.272480 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:21:00.275149 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 360/725] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:21:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:21:00.281726 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe83a9a1-0161-4da2-a847-f63180434bcf None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:00.349165 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 366/726] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:21:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 01:21:00.372716 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:00.373679 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3485cc-a764-4036-a938-a3c00fbdb5e3 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:00.422984 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:00.445317 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe83a9a1-0161-4da2-a847-f63180434bcf None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:00.508867 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7108fa8-6c13-4c9f-a937-f5be44b796c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:00.509533 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7108fa8-6c13-4c9f-a937-f5be44b796c9 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:00.509820 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7108fa8-6c13-4c9f-a937-f5be44b796c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:00.510095 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7108fa8-6c13-4c9f-a937-f5be44b796c9 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:00.510827 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7108fa8-6c13-4c9f-a937-f5be44b796c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:00.511136 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7108fa8-6c13-4c9f-a937-f5be44b796c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:00.512505 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7108fa8-6c13-4c9f-a937-f5be44b796c9 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:00.525355 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 361/727] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Aug 6 01:21:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:21:00.531830 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7108fa8-6c13-4c9f-a937-f5be44b796c9 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:00.560781 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3485cc-a764-4036-a938-a3c00fbdb5e3 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:00.624877 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a64fbd-3d2a-474f-a033-7b3e3361e5b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:00.626485 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a64fbd-3d2a-474f-a033-7b3e3361e5b1 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:00.627041 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a64fbd-3d2a-474f-a033-7b3e3361e5b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:00.627674 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a64fbd-3d2a-474f-a033-7b3e3361e5b1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:00.651600 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a64fbd-3d2a-474f-a033-7b3e3361e5b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:21:00.658167 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-52a64fbd-3d2a-474f-a033-7b3e3361e5b1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:21:00.658167 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:00.658167 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:21:00.658167 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:21:00.658167 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:21:00.658167 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:21:00.658167 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:21:00.658167 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:21:00.658167 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:21:00.658167 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:21:00.658167 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:21:00.658167 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:21:00.658167 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:21:00.658167 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:21:00.658167 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:21:00.658167 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:21:00.659150 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:21:00.659150 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:21:00.659150 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:21:00.659150 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:21:00.659150 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:21:00.659150 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:21:00.659150 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:21:00.659150 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:21:00.659150 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:21:00.659150 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:21:00.659150 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:21:00.659150 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:21:00.659150 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:21:00.659150 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:21:00.659150 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:21:00.659150 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:21:00.660049 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:21:00.660049 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:21:00.660049 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:21:00.660049 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:21:00.660049 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:21:00.660049 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:21:00.660049 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:21:00.660049 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:21:00.660049 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:21:00.660049 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:21:00.660049 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:21:00.660049 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:21:00.664525 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 367/728] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Aug 6 01:21:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:21:00.671455 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a64fbd-3d2a-474f-a033-7b3e3361e5b1 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:00.699276 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7108fa8-6c13-4c9f-a937-f5be44b796c9 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:00.707134 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:00.741250 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd37594-e17b-4599-86c0-e2993abe3450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:00.741960 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd37594-e17b-4599-86c0-e2993abe3450 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:00.742238 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd37594-e17b-4599-86c0-e2993abe3450 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:00.742569 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd37594-e17b-4599-86c0-e2993abe3450 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:00.751994 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd37594-e17b-4599-86c0-e2993abe3450 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:00.753538 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd37594-e17b-4599-86c0-e2993abe3450 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:00.767306 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-bfd37594-e17b-4599-86c0-e2993abe3450 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 06 01:21:00.767306 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:00.767306 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:21:00.767306 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:21:00.767306 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:21:00.767306 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:21:00.767306 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:21:00.767306 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:21:00.767306 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:21:00.767306 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:21:00.767306 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:21:00.767306 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:21:00.767306 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 01:21:00.767306 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 01:21:00.767306 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 01:21:00.767306 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 01:21:00.768312 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:21:00.768312 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:21:00.768312 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 01:21:00.768312 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:21:00.768312 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 01:21:00.768312 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:21:00.768312 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:21:00.768312 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:21:00.768312 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:21:00.768312 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:21:00.768312 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:21:00.768312 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:21:00.768312 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:21:00.768312 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:21:00.768312 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:21:00.768312 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:21:00.768312 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:21:00.768312 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:21:00.769288 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:21:00.769288 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:21:00.769288 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 01:21:00.769288 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 01:21:00.769288 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 01:21:00.769288 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 01:21:00.769288 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 01:21:00.769288 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 01:21:00.769288 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 06 01:21:00.769288 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:21:00.769288 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 362/729] 10.222.0.26 () {56 vars in 1117 bytes} [Sat Aug 6 01:21:00 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:21:00.775332 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd37594-e17b-4599-86c0-e2993abe3450 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:00.891935 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd37594-e17b-4599-86c0-e2993abe3450 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:00.908116 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:00.931557 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b57ab31-e885-481b-b703-08b2f4e50029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:00.932183 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b57ab31-e885-481b-b703-08b2f4e50029 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:00.932463 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b57ab31-e885-481b-b703-08b2f4e50029 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:00.932825 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b57ab31-e885-481b-b703-08b2f4e50029 None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:00.934061 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b57ab31-e885-481b-b703-08b2f4e50029 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:00.934369 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b57ab31-e885-481b-b703-08b2f4e50029 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:00.936139 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b57ab31-e885-481b-b703-08b2f4e50029 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:00.951294 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a64fbd-3d2a-474f-a033-7b3e3361e5b1 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:00.961716 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 363/730] 10.222.0.26 () {56 vars in 1149 bytes} [Sat Aug 6 01:21:00 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:21:00.962959 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b57ab31-e885-481b-b703-08b2f4e50029 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:00.998287 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:01.075903 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8bb256-fc26-4bb4-9900-909778083e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:01.076713 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8bb256-fc26-4bb4-9900-909778083e68 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:01.077115 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8bb256-fc26-4bb4-9900-909778083e68 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:01.077539 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8bb256-fc26-4bb4-9900-909778083e68 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:01.078426 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8bb256-fc26-4bb4-9900-909778083e68 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:01.078824 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8bb256-fc26-4bb4-9900-909778083e68 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:01.080368 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8bb256-fc26-4bb4-9900-909778083e68 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:01.086056 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b57ab31-e885-481b-b703-08b2f4e50029 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:01.107793 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 368/731] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Aug 6 01:21:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 438 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:21:01.126100 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8bb256-fc26-4bb4-9900-909778083e68 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:01.129594 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbae9988-205e-41a6-9624-4b55c3be96b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:01.130521 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbae9988-205e-41a6-9624-4b55c3be96b2 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:01.130976 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbae9988-205e-41a6-9624-4b55c3be96b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:01.131431 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbae9988-205e-41a6-9624-4b55c3be96b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:01.259757 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbae9988-205e-41a6-9624-4b55c3be96b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:01.262447 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbae9988-205e-41a6-9624-4b55c3be96b2 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:01.300409 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:01.398102 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 364/732] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Aug 6 01:21:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 06 01:21:01.398556 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744b7f57-38cd-417c-819e-a9222f7d8045 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:01.399169 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744b7f57-38cd-417c-819e-a9222f7d8045 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:01.399169 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744b7f57-38cd-417c-819e-a9222f7d8045 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:01.400583 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 365/733] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:21:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:21:01.402312 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744b7f57-38cd-417c-819e-a9222f7d8045 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:01.477753 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8bb256-fc26-4bb4-9900-909778083e68 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:01.518677 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663ab335-d326-4244-98cd-202ad2fdbd2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:01.519498 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663ab335-d326-4244-98cd-202ad2fdbd2f None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:01.519629 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663ab335-d326-4244-98cd-202ad2fdbd2f None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:01.519896 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663ab335-d326-4244-98cd-202ad2fdbd2f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:01.531305 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663ab335-d326-4244-98cd-202ad2fdbd2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:01.532813 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663ab335-d326-4244-98cd-202ad2fdbd2f None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:01.538603 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744b7f57-38cd-417c-819e-a9222f7d8045 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:01.540396 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:01.550770 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-663ab335-d326-4244-98cd-202ad2fdbd2f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:21:01.550770 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:01.550770 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:21:01.550770 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:21:01.550770 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:21:01.550770 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:21:01.550770 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:21:01.550770 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:21:01.550770 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:21:01.550770 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:21:01.550770 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:21:01.550770 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:21:01.550770 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:21:01.550770 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:21:01.550770 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:21:01.551852 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:21:01.551852 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:21:01.551852 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:21:01.551852 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:21:01.551852 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:21:01.551852 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:21:01.551852 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:21:01.551852 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:21:01.551852 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:21:01.551852 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:21:01.551852 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:21:01.551852 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:21:01.551852 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:21:01.551852 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:21:01.551852 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:21:01.551852 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:21:01.551852 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:21:01.552932 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:21:01.552932 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:21:01.552932 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:21:01.552932 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:21:01.560002 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 369/734] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:21:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 429 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:21:01.560002 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc344920-8463-48f2-b4c0-782b17121fdb None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:01.560002 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc344920-8463-48f2-b4c0-782b17121fdb None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:01.560002 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc344920-8463-48f2-b4c0-782b17121fdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:01.576976 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-bc344920-8463-48f2-b4c0-782b17121fdb None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:21:01.577942 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:01.579669 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-bc344920-8463-48f2-b4c0-782b17121fdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:21:01.591271 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804e2d18-504c-46e0-a28c-7079168ecac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:01.592521 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804e2d18-504c-46e0-a28c-7079168ecac5 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:01.592906 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804e2d18-504c-46e0-a28c-7079168ecac5 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:01.593311 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804e2d18-504c-46e0-a28c-7079168ecac5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:01.607237 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804e2d18-504c-46e0-a28c-7079168ecac5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:01.609800 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804e2d18-504c-46e0-a28c-7079168ecac5 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:01.628337 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-804e2d18-504c-46e0-a28c-7079168ecac5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:21:01.628337 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:01.628337 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:21:01.628337 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:21:01.628337 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:21:01.628337 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:21:01.628337 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:21:01.628337 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:21:01.628337 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:21:01.628337 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:21:01.628337 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:21:01.628337 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:21:01.628337 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:21:01.628337 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:21:01.628337 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:21:01.629298 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:21:01.629298 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:21:01.629298 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:21:01.629298 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:21:01.629298 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:21:01.629298 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:21:01.629298 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:21:01.629298 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:21:01.629298 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:21:01.629298 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:21:01.629298 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:21:01.629298 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:21:01.629298 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:21:01.629298 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:21:01.629298 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:21:01.629298 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:21:01.629298 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:21:01.630243 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:21:01.630243 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:21:01.630243 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:21:01.630243 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:21:01.633133 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 366/735] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:21:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:21:01.634159 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804e2d18-504c-46e0-a28c-7079168ecac5 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:01.661324 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 370/736] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:21:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:21:01.662641 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc344920-8463-48f2-b4c0-782b17121fdb None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:01.764483 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804e2d18-504c-46e0-a28c-7079168ecac5 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:01.773854 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc344920-8463-48f2-b4c0-782b17121fdb None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:01.799956 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f2a85f-ce74-497e-9cdf-b46a8aa665aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:01.800691 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f2a85f-ce74-497e-9cdf-b46a8aa665aa None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:01.801067 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f2a85f-ce74-497e-9cdf-b46a8aa665aa None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:01.801425 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f2a85f-ce74-497e-9cdf-b46a8aa665aa None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:01.802191 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f2a85f-ce74-497e-9cdf-b46a8aa665aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:01.802568 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f2a85f-ce74-497e-9cdf-b46a8aa665aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:01.804013 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f2a85f-ce74-497e-9cdf-b46a8aa665aa None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:01.806055 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:01.816600 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f1b6cd-7873-403f-9023-646341617c01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:01.821118 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 367/737] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Aug 6 01:21:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:01.821118 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f2a85f-ce74-497e-9cdf-b46a8aa665aa None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:01.821964 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f1b6cd-7873-403f-9023-646341617c01 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:01.822386 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f1b6cd-7873-403f-9023-646341617c01 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:01.822776 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f1b6cd-7873-403f-9023-646341617c01 None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:01.823702 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f1b6cd-7873-403f-9023-646341617c01 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:01.824304 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f1b6cd-7873-403f-9023-646341617c01 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:01.825839 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f1b6cd-7873-403f-9023-646341617c01 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:01.838608 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 371/738] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Aug 6 01:21:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:01.840368 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f1b6cd-7873-403f-9023-646341617c01 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:01.864220 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:01.926019 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f2a85f-ce74-497e-9cdf-b46a8aa665aa None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:01.951703 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f1b6cd-7873-403f-9023-646341617c01 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:01.961200 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732ed885-2ad6-4e72-8b51-8c52fdfdf55a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:01.961828 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732ed885-2ad6-4e72-8b51-8c52fdfdf55a None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:01.962112 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732ed885-2ad6-4e72-8b51-8c52fdfdf55a None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:01.962388 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732ed885-2ad6-4e72-8b51-8c52fdfdf55a None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:01.963091 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732ed885-2ad6-4e72-8b51-8c52fdfdf55a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:01.964651 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732ed885-2ad6-4e72-8b51-8c52fdfdf55a None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:01.984818 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 368/739] 10.222.0.26 () {60 vars in 1141 bytes} [Sat Aug 6 01:21:01 2022] POST /identity/v3/services => generated 251 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:21:01.985900 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732ed885-2ad6-4e72-8b51-8c52fdfdf55a None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:01.999574 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde837cd-d5f7-45be-9a0f-6a871d1d142a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:02.000442 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde837cd-d5f7-45be-9a0f-6a871d1d142a None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:02.000805 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde837cd-d5f7-45be-9a0f-6a871d1d142a None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:02.001223 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde837cd-d5f7-45be-9a0f-6a871d1d142a None admin] PATH_INFO: `/v3/users/swift` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:02.010612 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde837cd-d5f7-45be-9a0f-6a871d1d142a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:02.012871 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde837cd-d5f7-45be-9a0f-6a871d1d142a None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:02.031228 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-bde837cd-d5f7-45be-9a0f-6a871d1d142a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 06 01:21:02.031228 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:02.031228 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:21:02.031228 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:21:02.031228 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:21:02.031228 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:21:02.031228 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:21:02.031228 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:21:02.031228 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:21:02.031228 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:21:02.031228 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:21:02.031228 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:21:02.031228 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 01:21:02.031228 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 01:21:02.031228 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 01:21:02.031228 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 01:21:02.032401 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:21:02.032401 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:21:02.032401 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 01:21:02.032401 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:21:02.032401 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 01:21:02.032401 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:21:02.032401 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:21:02.032401 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:21:02.032401 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:21:02.032401 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:21:02.032401 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:21:02.032401 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:21:02.032401 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:21:02.032401 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:21:02.032401 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:21:02.032401 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:21:02.032401 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:21:02.032401 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:21:02.033398 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:21:02.033398 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:21:02.033398 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 01:21:02.033398 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 01:21:02.033398 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 01:21:02.033398 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 01:21:02.033398 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 01:21:02.033398 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 01:21:02.033398 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 06 01:21:02.033398 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:21:02.033398 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 372/740] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Aug 6 01:21:01 2022] GET /identity/v3/users/swift => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:21:02.040292 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde837cd-d5f7-45be-9a0f-6a871d1d142a None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:02.097882 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:02.104239 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:02.115883 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732ed885-2ad6-4e72-8b51-8c52fdfdf55a None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:02.154023 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1e613a-392b-4469-9472-2a91365cdd9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:02.154701 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1e613a-392b-4469-9472-2a91365cdd9f None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:02.154986 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1e613a-392b-4469-9472-2a91365cdd9f None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:02.155265 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1e613a-392b-4469-9472-2a91365cdd9f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:02.162836 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1e613a-392b-4469-9472-2a91365cdd9f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:02.164298 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1e613a-392b-4469-9472-2a91365cdd9f None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:02.177119 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-5d1e613a-392b-4469-9472-2a91365cdd9f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:21:02.177119 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:02.177119 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:21:02.177119 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:21:02.177119 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:21:02.177119 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:21:02.177119 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:21:02.177119 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:21:02.177119 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:21:02.177119 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:21:02.177119 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:21:02.177119 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:21:02.177119 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 01:21:02.177119 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 01:21:02.177119 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 01:21:02.179220 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 01:21:02.179220 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:21:02.179220 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:21:02.179220 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:21:02.179220 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:21:02.179220 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:21:02.179220 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:21:02.179220 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:21:02.179220 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:21:02.179220 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:21:02.179220 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:21:02.179220 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:21:02.179220 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:21:02.179220 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:21:02.179220 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:21:02.179220 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:21:02.179220 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:21:02.181388 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 01:21:02.181388 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 01:21:02.181388 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 01:21:02.181388 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 01:21:02.181388 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 01:21:02.181388 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 01:21:02.181388 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:21:02.181388 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:21:02.182409 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde837cd-d5f7-45be-9a0f-6a871d1d142a None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:02.185304 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 369/741] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Aug 6 01:21:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:21:02.190099 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1e613a-392b-4469-9472-2a91365cdd9f None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:02.219212 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e79ccf-3dee-4658-b3ef-a95cc263c73d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:02.219853 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e79ccf-3dee-4658-b3ef-a95cc263c73d None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:02.220188 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e79ccf-3dee-4658-b3ef-a95cc263c73d None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:02.220469 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e79ccf-3dee-4658-b3ef-a95cc263c73d None admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:02.221321 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e79ccf-3dee-4658-b3ef-a95cc263c73d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:02.221631 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e79ccf-3dee-4658-b3ef-a95cc263c73d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:02.223686 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e79ccf-3dee-4658-b3ef-a95cc263c73d None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:02.247003 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 373/742] 10.222.0.26 () {56 vars in 1135 bytes} [Sat Aug 6 01:21:02 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:02.248344 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52af9e82-42dc-47de-90c8-4dfb30d93378 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:02.248633 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52af9e82-42dc-47de-90c8-4dfb30d93378 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:02.248912 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52af9e82-42dc-47de-90c8-4dfb30d93378 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:02.253452 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 374/743] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:21:02 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:21:02.255410 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52af9e82-42dc-47de-90c8-4dfb30d93378 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:02.351654 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1e613a-392b-4469-9472-2a91365cdd9f None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:02.387706 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dd9527-281d-4044-a4d9-d71e24dfc71f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:02.388341 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dd9527-281d-4044-a4d9-d71e24dfc71f None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:02.388624 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dd9527-281d-4044-a4d9-d71e24dfc71f None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:02.388905 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dd9527-281d-4044-a4d9-d71e24dfc71f None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:02.389779 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dd9527-281d-4044-a4d9-d71e24dfc71f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:02.390018 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dd9527-281d-4044-a4d9-d71e24dfc71f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:02.394520 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dd9527-281d-4044-a4d9-d71e24dfc71f None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:02.400233 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52af9e82-42dc-47de-90c8-4dfb30d93378 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:02.402719 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:02.419218 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 370/744] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Aug 6 01:21:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:02.421615 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131d9dc0-1d8f-40fb-ba35-94383d0bca0d None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:02.422378 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131d9dc0-1d8f-40fb-ba35-94383d0bca0d None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:02.422839 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131d9dc0-1d8f-40fb-ba35-94383d0bca0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:02.444696 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a36f4b-05bf-44f3-9981-0ce647c99122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:02.445533 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a36f4b-05bf-44f3-9981-0ce647c99122 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:02.445817 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a36f4b-05bf-44f3-9981-0ce647c99122 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:02.446150 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a36f4b-05bf-44f3-9981-0ce647c99122 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:02.446833 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-131d9dc0-1d8f-40fb-ba35-94383d0bca0d None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:21:02.448341 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-131d9dc0-1d8f-40fb-ba35-94383d0bca0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:21:02.464051 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:02.522903 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a36f4b-05bf-44f3-9981-0ce647c99122 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:02.524356 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a36f4b-05bf-44f3-9981-0ce647c99122 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:02.538321 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 371/745] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:21:02 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:21:02.539156 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131d9dc0-1d8f-40fb-ba35-94383d0bca0d None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:02.641353 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 375/746] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Aug 6 01:21:02 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 01:21:02.642884 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2735a03f-fa12-445e-9c86-d04ffd0edb4f None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:02.643170 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2735a03f-fa12-445e-9c86-d04ffd0edb4f None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:02.643444 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2735a03f-fa12-445e-9c86-d04ffd0edb4f None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:02.644792 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 376/747] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:21:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:21:02.649223 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2735a03f-fa12-445e-9c86-d04ffd0edb4f None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:02.652955 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:02.655720 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131d9dc0-1d8f-40fb-ba35-94383d0bca0d None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:02.694504 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d519521-a052-4057-8ed1-9913e3fd3b5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:02.695125 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d519521-a052-4057-8ed1-9913e3fd3b5d None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:02.695404 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d519521-a052-4057-8ed1-9913e3fd3b5d None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:02.695681 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d519521-a052-4057-8ed1-9913e3fd3b5d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:02.696467 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d519521-a052-4057-8ed1-9913e3fd3b5d None admin] RBAC: Adding query filter params (role.id=c9ce798acd4b482f8f391c6af97eb7a4, scope.project.id=d9d68d310bf24449a5fe20d2dc264a58, user.id=8e86f38590824894a3b960e6774ff8a9) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:02.696781 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d519521-a052-4057-8ed1-9913e3fd3b5d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:02.698265 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d519521-a052-4057-8ed1-9913e3fd3b5d None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:02.704010 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 372/748] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Aug 6 01:21:02 2022] GET /identity/v3/role_assignments?user.id=8e86f38590824894a3b960e6774ff8a9&scope.project.id=d9d68d310bf24449a5fe20d2dc264a58&role.id=c9ce798acd4b482f8f391c6af97eb7a4 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:02.704783 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d519521-a052-4057-8ed1-9913e3fd3b5d None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:02.719958 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:02.797257 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2735a03f-fa12-445e-9c86-d04ffd0edb4f None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:02.839683 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6b26f5-529e-4889-99b9-c0e695b99c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:02.840506 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6b26f5-529e-4889-99b9-c0e695b99c07 None admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:02.840842 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6b26f5-529e-4889-99b9-c0e695b99c07 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:02.841151 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6b26f5-529e-4889-99b9-c0e695b99c07 None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:02.841863 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6b26f5-529e-4889-99b9-c0e695b99c07 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:02.842411 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d519521-a052-4057-8ed1-9913e3fd3b5d None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:02.843312 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6b26f5-529e-4889-99b9-c0e695b99c07 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:02.863091 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 377/749] 10.222.0.26 () {60 vars in 1141 bytes} [Sat Aug 6 01:21:02 2022] POST /identity/v3/services => generated 240 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:21:02.864543 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaf5362-4721-4830-8909-f2dd186a5406 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:02.864836 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaf5362-4721-4830-8909-f2dd186a5406 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:02.865157 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaf5362-4721-4830-8909-f2dd186a5406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:02.898003 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-aeaf5362-4721-4830-8909-f2dd186a5406 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:21:02.899635 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-aeaf5362-4721-4830-8909-f2dd186a5406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:21:02.920777 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f54bbb-17af-4527-b7dd-b40cf011ecd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:02.921966 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f54bbb-17af-4527-b7dd-b40cf011ecd3 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:02.922366 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f54bbb-17af-4527-b7dd-b40cf011ecd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:02.922715 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f54bbb-17af-4527-b7dd-b40cf011ecd3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:02.939319 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f54bbb-17af-4527-b7dd-b40cf011ecd3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:02.941708 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f54bbb-17af-4527-b7dd-b40cf011ecd3 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:02.963900 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-40f54bbb-17af-4527-b7dd-b40cf011ecd3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:21:02.963900 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:02.963900 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:21:02.963900 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:21:02.963900 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:21:02.963900 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:21:02.963900 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:21:02.963900 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:21:02.963900 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:21:02.963900 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:21:02.963900 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:21:02.963900 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:21:02.963900 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:21:02.963900 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:21:02.963900 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:21:02.964873 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:21:02.964873 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:21:02.964873 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:21:02.964873 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:21:02.964873 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:21:02.964873 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:21:02.964873 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:21:02.964873 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:21:02.964873 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:21:02.964873 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:21:02.964873 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:21:02.964873 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:21:02.964873 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:21:02.964873 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:21:02.964873 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:21:02.964873 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:21:02.964873 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:21:02.965806 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:21:02.965806 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:21:02.965806 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:21:02.965806 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:21:02.965806 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 373/750] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:21:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:21:02.972119 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f54bbb-17af-4527-b7dd-b40cf011ecd3 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:02.973782 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:02.989551 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 378/751] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:21:02 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:21:03.019536 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaf5362-4721-4830-8909-f2dd186a5406 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:03.084455 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:03.145498 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaf5362-4721-4830-8909-f2dd186a5406 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:03.165862 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f54bbb-17af-4527-b7dd-b40cf011ecd3 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:03.190302 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30df064-8e9f-4b86-a63c-c62e8ccb0de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:03.191201 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30df064-8e9f-4b86-a63c-c62e8ccb0de4 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:03.191623 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30df064-8e9f-4b86-a63c-c62e8ccb0de4 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:03.192048 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30df064-8e9f-4b86-a63c-c62e8ccb0de4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:03.206113 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae26942-76ff-493d-83cf-7bdb2dcdae01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:03.206350 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30df064-8e9f-4b86-a63c-c62e8ccb0de4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:21:03.206955 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae26942-76ff-493d-83cf-7bdb2dcdae01 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:03.207358 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae26942-76ff-493d-83cf-7bdb2dcdae01 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:03.207726 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae26942-76ff-493d-83cf-7bdb2dcdae01 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:03.208532 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae26942-76ff-493d-83cf-7bdb2dcdae01 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:03.208941 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae26942-76ff-493d-83cf-7bdb2dcdae01 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:03.210440 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae26942-76ff-493d-83cf-7bdb2dcdae01 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:03.212759 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-d30df064-8e9f-4b86-a63c-c62e8ccb0de4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:21:03.212759 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:03.212759 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:21:03.212759 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:21:03.212759 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:21:03.212759 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:21:03.212759 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:21:03.212759 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:21:03.212759 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:21:03.212759 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:21:03.212759 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:21:03.212759 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:21:03.212759 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:21:03.212759 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:21:03.212759 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:21:03.212759 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:21:03.214635 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:21:03.214635 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:21:03.214635 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:21:03.214635 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:21:03.214635 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:21:03.214635 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:21:03.214635 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:21:03.214635 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:21:03.214635 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:21:03.214635 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:21:03.214635 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:21:03.214635 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:21:03.214635 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:21:03.214635 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:21:03.214635 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:21:03.214635 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:21:03.215664 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:21:03.215664 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:21:03.215664 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:21:03.215664 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:21:03.215664 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:21:03.215664 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:21:03.215664 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:21:03.215664 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:21:03.215664 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:21:03.215664 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:21:03.215664 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:21:03.215664 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:21:03.218486 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 379/752] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Aug 6 01:21:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:21:03.225293 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30df064-8e9f-4b86-a63c-c62e8ccb0de4 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:03.228761 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 374/753] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Aug 6 01:21:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:03.238569 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae26942-76ff-493d-83cf-7bdb2dcdae01 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:03.297650 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:03.375541 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae26942-76ff-493d-83cf-7bdb2dcdae01 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:03.397545 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:03.421862 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30df064-8e9f-4b86-a63c-c62e8ccb0de4 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:03.426544 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bb2ddc-ea80-4fa2-a0d1-f3db94e8676c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:03.427250 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bb2ddc-ea80-4fa2-a0d1-f3db94e8676c None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:03.427549 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bb2ddc-ea80-4fa2-a0d1-f3db94e8676c None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:03.427828 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bb2ddc-ea80-4fa2-a0d1-f3db94e8676c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:03.442416 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bb2ddc-ea80-4fa2-a0d1-f3db94e8676c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:03.443875 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bb2ddc-ea80-4fa2-a0d1-f3db94e8676c None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:03.463051 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-e9bb2ddc-ea80-4fa2-a0d1-f3db94e8676c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:21:03.463051 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:03.463051 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:21:03.463051 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:21:03.463051 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:21:03.463051 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:21:03.463051 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:21:03.463051 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:21:03.463051 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:21:03.463051 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:21:03.463051 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:21:03.463051 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:21:03.463051 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 01:21:03.463051 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 01:21:03.463051 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 01:21:03.464576 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 01:21:03.464576 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:21:03.464576 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:21:03.464576 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:21:03.464576 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:21:03.464576 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:21:03.464576 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:21:03.464576 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:21:03.464576 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:21:03.464576 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:21:03.464576 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:21:03.464576 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:21:03.464576 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:21:03.464576 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:21:03.464576 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:21:03.464576 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:21:03.464576 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:21:03.465532 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 01:21:03.465532 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 01:21:03.465532 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 01:21:03.465532 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 01:21:03.465532 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 01:21:03.465532 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 01:21:03.465532 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:21:03.465532 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:21:03.468400 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 375/754] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Aug 6 01:21:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:21:03.471561 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bb2ddc-ea80-4fa2-a0d1-f3db94e8676c None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:03.494931 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d818ee-63d7-4856-b144-871aec646d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:03.495610 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d818ee-63d7-4856-b144-871aec646d08 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:03.495900 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d818ee-63d7-4856-b144-871aec646d08 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:03.496180 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d818ee-63d7-4856-b144-871aec646d08 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:03.496966 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d818ee-63d7-4856-b144-871aec646d08 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:03.497302 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d818ee-63d7-4856-b144-871aec646d08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:03.498689 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d818ee-63d7-4856-b144-871aec646d08 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:03.520521 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 380/755] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Aug 6 01:21:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:21:03.525898 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d818ee-63d7-4856-b144-871aec646d08 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:03.608737 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bb2ddc-ea80-4fa2-a0d1-f3db94e8676c None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:03.670986 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca031cf-e62b-4f46-bd30-e15de0d17724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:03.671617 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca031cf-e62b-4f46-bd30-e15de0d17724 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:03.671958 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca031cf-e62b-4f46-bd30-e15de0d17724 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:03.672182 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca031cf-e62b-4f46-bd30-e15de0d17724 None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:03.672942 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca031cf-e62b-4f46-bd30-e15de0d17724 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:03.673283 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca031cf-e62b-4f46-bd30-e15de0d17724 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:03.674971 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca031cf-e62b-4f46-bd30-e15de0d17724 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:03.702077 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 376/756] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Aug 6 01:21:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:21:03.709901 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d818ee-63d7-4856-b144-871aec646d08 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:03.714920 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca031cf-e62b-4f46-bd30-e15de0d17724 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:03.728821 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:03.753221 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a8d193-bd93-4349-adf8-5095704c0456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:03.754099 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a8d193-bd93-4349-adf8-5095704c0456 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:03.754951 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a8d193-bd93-4349-adf8-5095704c0456 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:03.755371 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a8d193-bd93-4349-adf8-5095704c0456 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:03.768792 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a8d193-bd93-4349-adf8-5095704c0456 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:03.770647 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a8d193-bd93-4349-adf8-5095704c0456 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:03.786742 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-a2a8d193-bd93-4349-adf8-5095704c0456 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:21:03.786742 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:03.786742 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:21:03.786742 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:21:03.786742 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:21:03.786742 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:21:03.786742 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:21:03.786742 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:21:03.786742 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:21:03.786742 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:21:03.786742 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:21:03.786742 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:21:03.786742 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:21:03.786742 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:21:03.786742 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:21:03.787907 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:21:03.787907 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:21:03.787907 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:21:03.787907 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:21:03.787907 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:21:03.787907 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:21:03.787907 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:21:03.787907 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:21:03.787907 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:21:03.787907 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:21:03.787907 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:21:03.787907 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:21:03.787907 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:21:03.787907 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:21:03.787907 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:21:03.787907 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:21:03.787907 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:21:03.788990 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:21:03.788990 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:21:03.788990 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:21:03.788990 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:21:03.796123 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 381/757] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:21:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:21:03.796705 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:03.802031 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a8d193-bd93-4349-adf8-5095704c0456 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:03.960267 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a8d193-bd93-4349-adf8-5095704c0456 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:03.967104 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca031cf-e62b-4f46-bd30-e15de0d17724 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:03.999048 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c1eded-a555-4d10-b873-d31404019e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:04.000032 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c1eded-a555-4d10-b873-d31404019e99 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:04.000509 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c1eded-a555-4d10-b873-d31404019e99 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:04.000975 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c1eded-a555-4d10-b873-d31404019e99 None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:04.002059 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c1eded-a555-4d10-b873-d31404019e99 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:04.002558 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c1eded-a555-4d10-b873-d31404019e99 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:04.004140 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c1eded-a555-4d10-b873-d31404019e99 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:04.019522 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 382/758] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Aug 6 01:21:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:04.025856 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c1eded-a555-4d10-b873-d31404019e99 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:04.065439 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:04.125774 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b90340a-017c-4d97-bdb7-802570f9fd3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:04.126452 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b90340a-017c-4d97-bdb7-802570f9fd3a None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:04.126739 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b90340a-017c-4d97-bdb7-802570f9fd3a None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:04.127019 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b90340a-017c-4d97-bdb7-802570f9fd3a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:04.127823 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b90340a-017c-4d97-bdb7-802570f9fd3a None admin] RBAC: Adding query filter params (role.id=4559b9a0837249f895a13c70096f8d38, scope.project.id=d9d68d310bf24449a5fe20d2dc264a58, user.id=2b23f4478f49471fba6f43e6e4be99da) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:04.128140 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b90340a-017c-4d97-bdb7-802570f9fd3a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:04.129639 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b90340a-017c-4d97-bdb7-802570f9fd3a None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:04.143122 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c1eded-a555-4d10-b873-d31404019e99 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:04.147017 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 377/759] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Aug 6 01:21:03 2022] GET /identity/v3/role_assignments?user.id=2b23f4478f49471fba6f43e6e4be99da&scope.project.id=d9d68d310bf24449a5fe20d2dc264a58&role.id=4559b9a0837249f895a13c70096f8d38 => generated 259 bytes in 433 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:04.218711 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0283aff1-18d5-44f5-9faa-e279c8715808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:04.219339 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0283aff1-18d5-44f5-9faa-e279c8715808 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:04.219625 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0283aff1-18d5-44f5-9faa-e279c8715808 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:04.219906 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0283aff1-18d5-44f5-9faa-e279c8715808 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:04.228457 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0283aff1-18d5-44f5-9faa-e279c8715808 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:04.230144 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0283aff1-18d5-44f5-9faa-e279c8715808 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:04.247317 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-0283aff1-18d5-44f5-9faa-e279c8715808 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 06 01:21:04.247317 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:04.247317 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:21:04.247317 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:21:04.247317 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:21:04.247317 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:21:04.247317 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:21:04.247317 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:21:04.247317 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:21:04.247317 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:21:04.247317 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:21:04.247317 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:21:04.247317 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 01:21:04.247317 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 01:21:04.247317 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 01:21:04.247317 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 01:21:04.248305 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:21:04.248305 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:21:04.248305 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 01:21:04.248305 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:21:04.248305 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 01:21:04.248305 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:21:04.248305 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:21:04.248305 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:21:04.248305 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:21:04.248305 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:21:04.248305 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:21:04.248305 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:21:04.248305 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:21:04.248305 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:21:04.248305 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:21:04.248305 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:21:04.248305 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:21:04.248305 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:21:04.249294 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:21:04.249294 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:21:04.249294 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 01:21:04.249294 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 01:21:04.249294 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 01:21:04.249294 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 01:21:04.249294 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 01:21:04.249294 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 01:21:04.249294 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 06 01:21:04.249294 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:21:04.249294 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 383/760] 10.222.0.26 () {56 vars in 1093 bytes} [Sat Aug 6 01:21:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:21:04.256499 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b90340a-017c-4d97-bdb7-802570f9fd3a None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:04.260784 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643f05c8-6c35-4ed8-808e-fbf5158ca962 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:04.261091 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643f05c8-6c35-4ed8-808e-fbf5158ca962 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:04.261835 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643f05c8-6c35-4ed8-808e-fbf5158ca962 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:04.262981 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 384/761] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:21:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:21:04.276101 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2cda8b-08fc-489f-8557-afb300e17830 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:04.278880 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2cda8b-08fc-489f-8557-afb300e17830 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:04.280042 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2cda8b-08fc-489f-8557-afb300e17830 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:04.314954 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-ea2cda8b-08fc-489f-8557-afb300e17830 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:21:04.317332 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-ea2cda8b-08fc-489f-8557-afb300e17830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:21:04.414465 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 385/762] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:21:04 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 06 01:21:04.414676 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b90340a-017c-4d97-bdb7-802570f9fd3a None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:04.416309 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:04.452058 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacfd14c-f8d9-4c01-85b2-8ad214aeb153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:04.452970 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacfd14c-f8d9-4c01-85b2-8ad214aeb153 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:04.453527 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacfd14c-f8d9-4c01-85b2-8ad214aeb153 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:04.453955 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacfd14c-f8d9-4c01-85b2-8ad214aeb153 None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:04.454734 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2cda8b-08fc-489f-8557-afb300e17830 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:04.455133 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacfd14c-f8d9-4c01-85b2-8ad214aeb153 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:04.455594 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacfd14c-f8d9-4c01-85b2-8ad214aeb153 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:04.457528 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacfd14c-f8d9-4c01-85b2-8ad214aeb153 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:04.493854 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:04.502333 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 378/763] 10.222.0.26 () {56 vars in 1133 bytes} [Sat Aug 6 01:21:04 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:04.514213 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacfd14c-f8d9-4c01-85b2-8ad214aeb153 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:04.632594 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2cda8b-08fc-489f-8557-afb300e17830 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:04.704100 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b793a26f-67b1-4ec2-9ef7-30d470216e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:04.704755 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b793a26f-67b1-4ec2-9ef7-30d470216e57 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:04.705058 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b793a26f-67b1-4ec2-9ef7-30d470216e57 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:04.705346 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b793a26f-67b1-4ec2-9ef7-30d470216e57 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:04.705988 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b793a26f-67b1-4ec2-9ef7-30d470216e57 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:04.707371 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b793a26f-67b1-4ec2-9ef7-30d470216e57 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:04.722701 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-b793a26f-67b1-4ec2-9ef7-30d470216e57 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 06 01:21:04.722701 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:04.722701 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 01:21:04.722701 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 01:21:04.722701 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 01:21:04.722701 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 01:21:04.722701 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 01:21:04.722701 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 01:21:04.722701 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 06 01:21:04.722701 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:21:04.722701 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 01:21:04.722701 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:21:04.722701 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:04.722701 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:21:04.722701 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:21:04.722701 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:21:04.723663 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:21:04.723663 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:21:04.723663 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:21:04.723663 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:21:04.723663 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:21:04.723663 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:21:04.723663 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:21:04.723663 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 01:21:04.723663 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 01:21:04.723663 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 01:21:04.723663 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 01:21:04.723663 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:21:04.723663 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:21:04.723663 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:21:04.723663 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:21:04.723663 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:21:04.724510 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:21:04.724510 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:21:04.724510 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:21:04.724510 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:21:04.724510 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:21:04.724510 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:21:04.724510 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:21:04.724510 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:21:04.724510 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:21:04.724510 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:21:04.724510 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:21:04.724510 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 01:21:04.724510 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 01:21:04.724510 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 06 01:21:04.724510 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:21:04.726999 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacfd14c-f8d9-4c01-85b2-8ad214aeb153 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:04.729898 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 386/764] 10.222.0.26 () {56 vars in 1129 bytes} [Sat Aug 6 01:21:04 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:21:04.730924 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b793a26f-67b1-4ec2-9ef7-30d470216e57 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:04.794793 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42f1b51-85db-4df2-abb3-f0afc53b115b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:04.795367 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42f1b51-85db-4df2-abb3-f0afc53b115b None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:04.795655 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42f1b51-85db-4df2-abb3-f0afc53b115b None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:04.795937 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42f1b51-85db-4df2-abb3-f0afc53b115b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:04.816243 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42f1b51-85db-4df2-abb3-f0afc53b115b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:04.816529 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:04.817740 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42f1b51-85db-4df2-abb3-f0afc53b115b None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:04.839507 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b793a26f-67b1-4ec2-9ef7-30d470216e57 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:04.847843 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-a42f1b51-85db-4df2-abb3-f0afc53b115b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:21:04.847843 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:04.847843 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:21:04.847843 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:21:04.847843 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:21:04.847843 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:21:04.847843 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:21:04.847843 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:21:04.847843 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:21:04.847843 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:21:04.847843 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:21:04.847843 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:21:04.847843 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:21:04.847843 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:21:04.847843 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:21:04.849995 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:21:04.849995 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:21:04.849995 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:21:04.849995 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:21:04.849995 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:21:04.849995 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:21:04.849995 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:21:04.849995 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:21:04.849995 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:21:04.849995 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:21:04.849995 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:21:04.849995 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:21:04.849995 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:21:04.849995 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:21:04.849995 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:21:04.849995 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:21:04.849995 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:21:04.852395 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:21:04.852395 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:21:04.852395 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:21:04.852395 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:21:04.857119 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 379/765] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:21:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:21:04.861831 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42f1b51-85db-4df2-abb3-f0afc53b115b None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:04.879914 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ca070a-9457-47f4-bc22-7e9b4edd77da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:04.880558 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ca070a-9457-47f4-bc22-7e9b4edd77da None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:04.880841 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ca070a-9457-47f4-bc22-7e9b4edd77da None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:04.881237 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ca070a-9457-47f4-bc22-7e9b4edd77da None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:04.881914 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ca070a-9457-47f4-bc22-7e9b4edd77da None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:04.882217 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ca070a-9457-47f4-bc22-7e9b4edd77da None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:04.883577 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ca070a-9457-47f4-bc22-7e9b4edd77da None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:04.889389 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 387/766] 10.222.0.26 () {56 vars in 1124 bytes} [Sat Aug 6 01:21:04 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:04.892393 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:04.894774 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ca070a-9457-47f4-bc22-7e9b4edd77da None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:05.075719 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ca070a-9457-47f4-bc22-7e9b4edd77da None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:05.078954 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42f1b51-85db-4df2-abb3-f0afc53b115b None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:05.124157 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a07554c-f712-4911-828d-118af389d88c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:05.124782 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a07554c-f712-4911-828d-118af389d88c None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:05.125088 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a07554c-f712-4911-828d-118af389d88c None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:05.125373 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a07554c-f712-4911-828d-118af389d88c None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:05.126078 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a07554c-f712-4911-828d-118af389d88c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:05.126391 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a07554c-f712-4911-828d-118af389d88c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:05.127760 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a07554c-f712-4911-828d-118af389d88c None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:05.129663 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7a6d51-a97d-4203-aa6e-994e08c229a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:05.130821 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7a6d51-a97d-4203-aa6e-994e08c229a4 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:05.131101 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7a6d51-a97d-4203-aa6e-994e08c229a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:05.131382 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7a6d51-a97d-4203-aa6e-994e08c229a4 None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:05.132101 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7a6d51-a97d-4203-aa6e-994e08c229a4 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:05.132405 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7a6d51-a97d-4203-aa6e-994e08c229a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:05.133790 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7a6d51-a97d-4203-aa6e-994e08c229a4 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:05.140810 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 388/767] 10.222.0.26 () {56 vars in 1124 bytes} [Sat Aug 6 01:21:04 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:21:05.141256 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 380/768] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Aug 6 01:21:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:21:05.142039 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1489a1-0aa7-49f9-8fd7-0a8c2843353c None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:05.142326 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1489a1-0aa7-49f9-8fd7-0a8c2843353c None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:05.142601 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1489a1-0aa7-49f9-8fd7-0a8c2843353c None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:05.143902 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 389/769] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:21:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:21:05.149082 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a07554c-f712-4911-828d-118af389d88c None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:05.149414 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca24dcb-a89c-467b-ad55-447e821fb2a6 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:05.149818 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca24dcb-a89c-467b-ad55-447e821fb2a6 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:05.150097 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca24dcb-a89c-467b-ad55-447e821fb2a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:05.174910 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-aca24dcb-a89c-467b-ad55-447e821fb2a6 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:21:05.176751 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-aca24dcb-a89c-467b-ad55-447e821fb2a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:21:05.179695 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:05.228731 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:05.268261 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 390/770] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:21:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:21:05.269267 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca24dcb-a89c-467b-ad55-447e821fb2a6 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:05.269998 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a07554c-f712-4911-828d-118af389d88c None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:05.308499 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4762c17f-3fb3-477e-9b92-7adbaadf770b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:05.309423 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4762c17f-3fb3-477e-9b92-7adbaadf770b None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:05.309884 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4762c17f-3fb3-477e-9b92-7adbaadf770b None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:05.310307 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4762c17f-3fb3-477e-9b92-7adbaadf770b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:05.311325 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4762c17f-3fb3-477e-9b92-7adbaadf770b None admin] RBAC: Adding query filter params (interface=public, service_id=e853c0b25550464fb46559611cc2e19a, region_id=RegionOne) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:05.311781 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4762c17f-3fb3-477e-9b92-7adbaadf770b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:05.313425 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4762c17f-3fb3-477e-9b92-7adbaadf770b None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:05.322381 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 381/771] 10.222.0.26 () {56 vars in 1251 bytes} [Sat Aug 6 01:21:05 2022] GET /identity/v3/endpoints?service_id=e853c0b25550464fb46559611cc2e19a&interface=public®ion_id=RegionOne => generated 194 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:05.323930 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4762c17f-3fb3-477e-9b92-7adbaadf770b None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:05.373833 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca24dcb-a89c-467b-ad55-447e821fb2a6 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:05.411532 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5fb760-cc26-49a8-a977-ed3cbc42869d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:05.412342 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5fb760-cc26-49a8-a977-ed3cbc42869d None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:05.412742 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5fb760-cc26-49a8-a977-ed3cbc42869d None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:05.413140 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5fb760-cc26-49a8-a977-ed3cbc42869d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:05.421338 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5fb760-cc26-49a8-a977-ed3cbc42869d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:05.422940 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5fb760-cc26-49a8-a977-ed3cbc42869d None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:05.424901 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:05.437128 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-6e5fb760-cc26-49a8-a977-ed3cbc42869d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:21:05.437128 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:05.437128 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:21:05.437128 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:21:05.437128 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:21:05.437128 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:21:05.437128 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:21:05.437128 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:21:05.437128 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:21:05.437128 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:21:05.437128 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:21:05.437128 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:21:05.437128 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 01:21:05.437128 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 01:21:05.437128 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 01:21:05.438095 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 01:21:05.438095 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:21:05.438095 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:21:05.438095 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:21:05.438095 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:21:05.438095 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:21:05.438095 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:21:05.438095 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:21:05.438095 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:21:05.438095 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:21:05.438095 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:21:05.438095 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:21:05.438095 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:21:05.438095 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:21:05.438095 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:21:05.438095 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:21:05.438095 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:21:05.438978 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 01:21:05.438978 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 01:21:05.438978 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 01:21:05.438978 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 01:21:05.438978 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 01:21:05.438978 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 01:21:05.438978 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:21:05.438978 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:21:05.438978 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 391/772] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Aug 6 01:21:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:21:05.439459 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5fb760-cc26-49a8-a977-ed3cbc42869d None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:05.449839 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4762c17f-3fb3-477e-9b92-7adbaadf770b None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:05.488812 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8de19a-b2cf-46dc-b016-b451884d427d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:05.489900 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8de19a-b2cf-46dc-b016-b451884d427d None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:05.490364 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8de19a-b2cf-46dc-b016-b451884d427d None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:05.490846 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8de19a-b2cf-46dc-b016-b451884d427d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:05.493968 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:05.505073 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8de19a-b2cf-46dc-b016-b451884d427d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:21:05.511527 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-ea8de19a-b2cf-46dc-b016-b451884d427d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 06 01:21:05.511527 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:05.511527 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:21:05.511527 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:21:05.511527 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:21:05.511527 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:21:05.511527 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:21:05.511527 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:21:05.511527 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:21:05.511527 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:21:05.511527 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:21:05.511527 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:21:05.511527 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:21:05.511527 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:21:05.511527 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:21:05.511527 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:21:05.512454 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:21:05.512454 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:21:05.512454 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:21:05.512454 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:21:05.512454 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:21:05.512454 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:21:05.512454 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:21:05.512454 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:21:05.512454 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:21:05.512454 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:21:05.512454 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:21:05.512454 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:21:05.512454 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:21:05.512454 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:21:05.512454 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:21:05.512454 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:21:05.513324 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:21:05.513324 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:21:05.513324 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:21:05.513324 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:21:05.513324 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:21:05.513324 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:21:05.513324 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:21:05.513324 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:21:05.513324 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:21:05.513324 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:21:05.513324 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 06 01:21:05.513324 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:21:05.516011 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 382/773] 10.222.0.26 () {56 vars in 1102 bytes} [Sat Aug 6 01:21:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:21:05.517182 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8de19a-b2cf-46dc-b016-b451884d427d None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:05.551993 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5fb760-cc26-49a8-a977-ed3cbc42869d None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:05.587094 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4074c0e-1eac-4a5e-8565-5b3680c5d3f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:05.587845 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4074c0e-1eac-4a5e-8565-5b3680c5d3f9 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:05.588225 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4074c0e-1eac-4a5e-8565-5b3680c5d3f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:05.588569 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4074c0e-1eac-4a5e-8565-5b3680c5d3f9 None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:05.590097 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4074c0e-1eac-4a5e-8565-5b3680c5d3f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:05.591479 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4074c0e-1eac-4a5e-8565-5b3680c5d3f9 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:05.596599 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 392/774] 10.222.0.26 () {56 vars in 1087 bytes} [Sat Aug 6 01:21:05 2022] GET /identity/v3/services => generated 772 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:05.597403 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4074c0e-1eac-4a5e-8565-5b3680c5d3f9 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:05.631066 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8de19a-b2cf-46dc-b016-b451884d427d None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:05.680837 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:05.682909 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e3ed39-ca17-471b-b800-c4d6d23fa1b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:05.683471 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e3ed39-ca17-471b-b800-c4d6d23fa1b6 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:05.683809 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e3ed39-ca17-471b-b800-c4d6d23fa1b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:05.684013 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e3ed39-ca17-471b-b800-c4d6d23fa1b6 None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:05.684885 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e3ed39-ca17-471b-b800-c4d6d23fa1b6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:05.685123 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e3ed39-ca17-471b-b800-c4d6d23fa1b6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:05.686600 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e3ed39-ca17-471b-b800-c4d6d23fa1b6 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:05.712523 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 383/775] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Aug 6 01:21:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:05.712523 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1222f26f-2052-42d5-a533-44513d5735ba None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:05.712523 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1222f26f-2052-42d5-a533-44513d5735ba None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:05.712523 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1222f26f-2052-42d5-a533-44513d5735ba None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:05.713416 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 384/776] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:21:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:21:05.715061 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1222f26f-2052-42d5-a533-44513d5735ba None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:05.742703 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4074c0e-1eac-4a5e-8565-5b3680c5d3f9 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:05.787105 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572ea233-7403-4e01-aa71-4c60564338f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:05.787734 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572ea233-7403-4e01-aa71-4c60564338f8 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:05.788176 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572ea233-7403-4e01-aa71-4c60564338f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:05.788457 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572ea233-7403-4e01-aa71-4c60564338f8 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:05.789144 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:05.789554 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572ea233-7403-4e01-aa71-4c60564338f8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:05.789868 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572ea233-7403-4e01-aa71-4c60564338f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:05.791390 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572ea233-7403-4e01-aa71-4c60564338f8 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:05.802536 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 393/777] 10.222.0.26 () {56 vars in 1103 bytes} [Sat Aug 6 01:21:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:05.804051 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e820d6-c408-45b1-a8b8-a341f92f8c73 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:05.804336 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e820d6-c408-45b1-a8b8-a341f92f8c73 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:05.804613 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e820d6-c408-45b1-a8b8-a341f92f8c73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:05.827467 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-f0e820d6-c408-45b1-a8b8-a341f92f8c73 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:21:05.828694 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-f0e820d6-c408-45b1-a8b8-a341f92f8c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:21:05.849250 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1222f26f-2052-42d5-a533-44513d5735ba None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:05.895274 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e57429b-df61-4309-8f15-d5f8ea16d350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:05.895903 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e57429b-df61-4309-8f15-d5f8ea16d350 None admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:05.896223 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e57429b-df61-4309-8f15-d5f8ea16d350 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:05.896517 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e57429b-df61-4309-8f15-d5f8ea16d350 None admin] PATH_INFO: `/v3/projects/d9d68d310bf24449a5fe20d2dc264a58/users/b87adc1d5bc240e99f95f6595b350be4/roles/4559b9a0837249f895a13c70096f8d38` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:05.899479 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e57429b-df61-4309-8f15-d5f8ea16d350 None admin] RBAC: Authorizing `identity:create_grant(project_id=d9d68d310bf24449a5fe20d2dc264a58, user_id=b87adc1d5bc240e99f95f6595b350be4, role_id=4559b9a0837249f895a13c70096f8d38)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:05.902934 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e57429b-df61-4309-8f15-d5f8ea16d350 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:05.916553 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 394/778] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:21:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:21:05.922212 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e820d6-c408-45b1-a8b8-a341f92f8c73 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:05.928104 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 385/779] 10.222.0.26 () {58 vars in 1439 bytes} [Sat Aug 6 01:21:05 2022] PUT /identity/v3/projects/d9d68d310bf24449a5fe20d2dc264a58/users/b87adc1d5bc240e99f95f6595b350be4/roles/4559b9a0837249f895a13c70096f8d38 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:21:05.946424 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e57429b-df61-4309-8f15-d5f8ea16d350 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:06.010247 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:06.063495 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e57429b-df61-4309-8f15-d5f8ea16d350 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:06.073830 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e820d6-c408-45b1-a8b8-a341f92f8c73 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:06.093101 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:06.120691 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cbfd4f-ea4c-4fac-a7f0-d16b48c3b939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:06.121783 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cbfd4f-ea4c-4fac-a7f0-d16b48c3b939 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:06.122110 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be333358-a28b-48eb-a375-15b9c1061099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:06.122417 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cbfd4f-ea4c-4fac-a7f0-d16b48c3b939 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:06.122932 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cbfd4f-ea4c-4fac-a7f0-d16b48c3b939 None admin] PATH_INFO: `/v3/services/network` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:06.123048 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be333358-a28b-48eb-a375-15b9c1061099 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:06.123417 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be333358-a28b-48eb-a375-15b9c1061099 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:06.123792 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be333358-a28b-48eb-a375-15b9c1061099 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:06.124136 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cbfd4f-ea4c-4fac-a7f0-d16b48c3b939 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:06.125930 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cbfd4f-ea4c-4fac-a7f0-d16b48c3b939 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:06.136963 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be333358-a28b-48eb-a375-15b9c1061099 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:06.138644 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be333358-a28b-48eb-a375-15b9c1061099 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:06.142143 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-e3cbfd4f-ea4c-4fac-a7f0-d16b48c3b939 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 06 01:21:06.142143 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:06.142143 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 01:21:06.142143 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 01:21:06.142143 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 01:21:06.142143 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 01:21:06.142143 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 01:21:06.142143 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 01:21:06.142143 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 06 01:21:06.142143 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:21:06.142143 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 01:21:06.142143 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:21:06.142143 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:06.142143 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:21:06.142143 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:21:06.142143 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:21:06.142143 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:21:06.143444 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:21:06.143444 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:21:06.143444 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:21:06.143444 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:21:06.143444 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:21:06.143444 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:21:06.143444 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 01:21:06.143444 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 01:21:06.143444 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 01:21:06.143444 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 01:21:06.143444 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:21:06.143444 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:21:06.143444 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:21:06.143444 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:21:06.143444 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:21:06.143444 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:21:06.144504 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:21:06.144504 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:21:06.144504 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:21:06.144504 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:21:06.144504 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:21:06.144504 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:21:06.144504 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:21:06.144504 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:21:06.144504 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:21:06.144504 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:21:06.144504 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 01:21:06.144504 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 01:21:06.144504 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 06 01:21:06.144504 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:21:06.147297 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 386/780] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Aug 6 01:21:05 2022] GET /identity/v3/services/network => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:21:06.153982 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cbfd4f-ea4c-4fac-a7f0-d16b48c3b939 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:06.164958 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-be333358-a28b-48eb-a375-15b9c1061099 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:21:06.164958 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:06.164958 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:21:06.164958 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:21:06.164958 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:21:06.164958 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:21:06.164958 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:21:06.164958 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:21:06.164958 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:21:06.164958 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:21:06.164958 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:21:06.164958 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:21:06.164958 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:21:06.164958 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:21:06.164958 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:21:06.166690 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:21:06.166690 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:21:06.166690 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:21:06.166690 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:21:06.166690 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:21:06.166690 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:21:06.166690 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:21:06.166690 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:21:06.166690 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:21:06.166690 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:21:06.166690 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:21:06.166690 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:21:06.166690 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:21:06.166690 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:21:06.166690 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:21:06.166690 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:21:06.166690 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:21:06.167672 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:21:06.167672 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:21:06.167672 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:21:06.167672 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:21:06.170410 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 395/781] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:21:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:21:06.176348 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be333358-a28b-48eb-a375-15b9c1061099 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:06.269095 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cbfd4f-ea4c-4fac-a7f0-d16b48c3b939 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:06.292949 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be333358-a28b-48eb-a375-15b9c1061099 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:06.301161 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:06.308436 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923d99ea-6140-4b9f-b3f4-0b1edc8478db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:06.309633 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923d99ea-6140-4b9f-b3f4-0b1edc8478db None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:06.310032 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923d99ea-6140-4b9f-b3f4-0b1edc8478db None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:06.310384 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923d99ea-6140-4b9f-b3f4-0b1edc8478db None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:06.311135 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923d99ea-6140-4b9f-b3f4-0b1edc8478db None admin] RBAC: Adding query filter params (name=network) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:06.311531 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923d99ea-6140-4b9f-b3f4-0b1edc8478db None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:06.313076 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923d99ea-6140-4b9f-b3f4-0b1edc8478db None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:06.319988 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 387/782] 10.222.0.26 () {56 vars in 1112 bytes} [Sat Aug 6 01:21:06 2022] GET /identity/v3/services?name=network => generated 124 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:21:06.327570 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923d99ea-6140-4b9f-b3f4-0b1edc8478db None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:06.361750 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccfd875-ee39-4d5c-9ddc-1e4e80ae300a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:06.362819 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccfd875-ee39-4d5c-9ddc-1e4e80ae300a None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:06.363306 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccfd875-ee39-4d5c-9ddc-1e4e80ae300a None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:06.363783 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccfd875-ee39-4d5c-9ddc-1e4e80ae300a None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:06.364891 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccfd875-ee39-4d5c-9ddc-1e4e80ae300a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:06.366448 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccfd875-ee39-4d5c-9ddc-1e4e80ae300a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:06.369559 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccfd875-ee39-4d5c-9ddc-1e4e80ae300a None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:06.399044 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 396/783] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Aug 6 01:21:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:06.405505 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccfd875-ee39-4d5c-9ddc-1e4e80ae300a None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:06.417722 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:06.502136 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923d99ea-6140-4b9f-b3f4-0b1edc8478db None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:06.549476 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa5926a-7c28-4f6d-ae8f-9bf5e5222d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:06.550451 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa5926a-7c28-4f6d-ae8f-9bf5e5222d7a None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:06.550957 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa5926a-7c28-4f6d-ae8f-9bf5e5222d7a None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:06.551435 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa5926a-7c28-4f6d-ae8f-9bf5e5222d7a None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:06.552829 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa5926a-7c28-4f6d-ae8f-9bf5e5222d7a None admin] RBAC: Adding query filter params (type=network) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:06.553616 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa5926a-7c28-4f6d-ae8f-9bf5e5222d7a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:06.555872 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa5926a-7c28-4f6d-ae8f-9bf5e5222d7a None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:06.564439 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccfd875-ee39-4d5c-9ddc-1e4e80ae300a None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:06.564694 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 388/784] 10.222.0.26 () {56 vars in 1112 bytes} [Sat Aug 6 01:21:06 2022] GET /identity/v3/services?type=network => generated 350 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:06.570292 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0adb87-67b7-4f4a-9590-e136d492b033 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:06.570292 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0adb87-67b7-4f4a-9590-e136d492b033 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:06.570292 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0adb87-67b7-4f4a-9590-e136d492b033 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:06.571518 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 389/785] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:21:06 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:21:06.572807 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0adb87-67b7-4f4a-9590-e136d492b033 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:06.615336 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1fd50f-082a-4f3f-a57d-8b7deb4d6203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:06.616482 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1fd50f-082a-4f3f-a57d-8b7deb4d6203 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:06.617229 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1fd50f-082a-4f3f-a57d-8b7deb4d6203 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:06.617793 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1fd50f-082a-4f3f-a57d-8b7deb4d6203 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:06.626538 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1fd50f-082a-4f3f-a57d-8b7deb4d6203 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:06.628075 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1fd50f-082a-4f3f-a57d-8b7deb4d6203 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:06.641263 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-0b1fd50f-082a-4f3f-a57d-8b7deb4d6203 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 06 01:21:06.641263 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:06.641263 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:21:06.641263 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:21:06.641263 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:21:06.641263 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:21:06.641263 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:21:06.641263 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:21:06.641263 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:21:06.641263 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:21:06.641263 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:21:06.641263 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:21:06.641263 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 01:21:06.641263 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 01:21:06.641263 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 01:21:06.641263 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 01:21:06.642251 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:21:06.642251 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:21:06.642251 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 01:21:06.642251 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:21:06.642251 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 01:21:06.642251 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:21:06.642251 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:21:06.642251 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:21:06.642251 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:21:06.642251 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:21:06.642251 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:21:06.642251 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:21:06.642251 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:21:06.642251 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:21:06.642251 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:21:06.642251 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:21:06.642251 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:21:06.642251 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:21:06.643196 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:21:06.643196 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:21:06.643196 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 01:21:06.643196 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 01:21:06.643196 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 01:21:06.643196 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 01:21:06.643196 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 01:21:06.643196 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 01:21:06.643196 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 06 01:21:06.643196 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:21:06.647365 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 397/786] 10.222.0.26 () {56 vars in 1117 bytes} [Sat Aug 6 01:21:06 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:21:06.648863 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3313568-08e7-436c-a517-ea7af66ce2a1 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:06.649178 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3313568-08e7-436c-a517-ea7af66ce2a1 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:06.649529 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3313568-08e7-436c-a517-ea7af66ce2a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:06.669211 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-d3313568-08e7-436c-a517-ea7af66ce2a1 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:21:06.670568 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-d3313568-08e7-436c-a517-ea7af66ce2a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:21:06.693751 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:06.704309 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0adb87-67b7-4f4a-9590-e136d492b033 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:06.715661 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:06.744130 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbe1f64-a51b-464b-95f9-231d36f46b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:06.745219 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbe1f64-a51b-464b-95f9-231d36f46b23 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:06.745966 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbe1f64-a51b-464b-95f9-231d36f46b23 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:06.746512 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbe1f64-a51b-464b-95f9-231d36f46b23 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:06.747474 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbe1f64-a51b-464b-95f9-231d36f46b23 None admin] RBAC: Adding query filter params (interface=public, service_id=8ae12aba232c4a65a9792d620380e380, region_id=RegionOne) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:06.747986 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbe1f64-a51b-464b-95f9-231d36f46b23 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:06.749580 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbe1f64-a51b-464b-95f9-231d36f46b23 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:06.753518 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 398/787] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:21:06 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:21:06.754432 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3313568-08e7-436c-a517-ea7af66ce2a1 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:06.755771 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 390/788] 10.222.0.26 () {56 vars in 1251 bytes} [Sat Aug 6 01:21:06 2022] GET /identity/v3/endpoints?service_id=8ae12aba232c4a65a9792d620380e380&interface=public®ion_id=RegionOne => generated 194 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:06.766090 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbe1f64-a51b-464b-95f9-231d36f46b23 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:06.868546 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3313568-08e7-436c-a517-ea7af66ce2a1 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:06.907766 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ff9283-b725-4262-bdf0-dacee1c12156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:06.908402 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ff9283-b725-4262-bdf0-dacee1c12156 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:06.908689 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ff9283-b725-4262-bdf0-dacee1c12156 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:06.908965 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ff9283-b725-4262-bdf0-dacee1c12156 None admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:06.909797 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ff9283-b725-4262-bdf0-dacee1c12156 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:06.910106 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ff9283-b725-4262-bdf0-dacee1c12156 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:06.910508 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbe1f64-a51b-464b-95f9-231d36f46b23 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:06.911531 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ff9283-b725-4262-bdf0-dacee1c12156 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:06.935287 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 399/789] 10.222.0.26 () {56 vars in 1149 bytes} [Sat Aug 6 01:21:06 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:06.936217 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ff9283-b725-4262-bdf0-dacee1c12156 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:06.949880 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41f6e8d-fe5a-4816-bb77-e0efde42418f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:06.950569 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41f6e8d-fe5a-4816-bb77-e0efde42418f None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:06.950856 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41f6e8d-fe5a-4816-bb77-e0efde42418f None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:06.951132 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41f6e8d-fe5a-4816-bb77-e0efde42418f None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:06.951835 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41f6e8d-fe5a-4816-bb77-e0efde42418f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:06.952128 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:06.953294 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41f6e8d-fe5a-4816-bb77-e0efde42418f None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:06.959136 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 391/790] 10.222.0.26 () {56 vars in 1087 bytes} [Sat Aug 6 01:21:06 2022] GET /identity/v3/services => generated 772 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:06.959931 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41f6e8d-fe5a-4816-bb77-e0efde42418f None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:07.003415 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:07.050006 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ff9283-b725-4262-bdf0-dacee1c12156 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:07.078102 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41f6e8d-fe5a-4816-bb77-e0efde42418f None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:07.088399 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62761729-ae9c-438f-b55b-59b46d8061c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:07.089975 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62761729-ae9c-438f-b55b-59b46d8061c6 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:07.090473 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62761729-ae9c-438f-b55b-59b46d8061c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:07.090982 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62761729-ae9c-438f-b55b-59b46d8061c6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:07.102951 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62761729-ae9c-438f-b55b-59b46d8061c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:21:07.109082 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-62761729-ae9c-438f-b55b-59b46d8061c6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:21:07.109082 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:07.109082 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:21:07.109082 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:21:07.109082 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:21:07.109082 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:21:07.109082 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:21:07.109082 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:21:07.109082 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:21:07.109082 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:21:07.109082 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:21:07.109082 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:21:07.109082 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:21:07.109082 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:21:07.109082 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:21:07.109082 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:21:07.110046 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:21:07.110046 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:21:07.110046 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:21:07.110046 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:21:07.110046 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:21:07.110046 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:21:07.110046 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:21:07.110046 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:21:07.110046 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:21:07.110046 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:21:07.110046 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:21:07.110046 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:21:07.110046 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:21:07.110046 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:21:07.110046 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:21:07.110046 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:21:07.110889 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:21:07.110889 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:21:07.110889 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:21:07.110889 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:21:07.110889 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:21:07.110889 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:21:07.110889 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:21:07.110889 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:21:07.110889 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:21:07.110889 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:21:07.110889 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:21:07.110889 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:21:07.113260 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 400/791] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Aug 6 01:21:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:21:07.116805 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97641f2-9187-4b1b-9140-f3543b5a4a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:07.121151 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97641f2-9187-4b1b-9140-f3543b5a4a10 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:07.121151 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97641f2-9187-4b1b-9140-f3543b5a4a10 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:07.121151 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97641f2-9187-4b1b-9140-f3543b5a4a10 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:07.122491 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62761729-ae9c-438f-b55b-59b46d8061c6 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:07.147368 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97641f2-9187-4b1b-9140-f3543b5a4a10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:07.149912 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97641f2-9187-4b1b-9140-f3543b5a4a10 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:07.181684 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-e97641f2-9187-4b1b-9140-f3543b5a4a10 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:21:07.181684 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:07.181684 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:21:07.181684 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:21:07.181684 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:21:07.181684 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:21:07.181684 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:21:07.181684 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:21:07.181684 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:21:07.181684 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:21:07.181684 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:21:07.181684 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:21:07.181684 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:21:07.181684 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:21:07.181684 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:21:07.183649 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:21:07.183649 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:21:07.183649 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:21:07.183649 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:21:07.183649 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:21:07.183649 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:21:07.183649 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:21:07.183649 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:21:07.183649 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:21:07.183649 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:21:07.183649 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:21:07.183649 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:21:07.183649 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:21:07.183649 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:21:07.183649 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:21:07.183649 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:21:07.183649 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:21:07.184859 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:21:07.184859 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:21:07.184859 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:21:07.184859 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:21:07.190107 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 392/792] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:21:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:21:07.195705 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97641f2-9187-4b1b-9140-f3543b5a4a10 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:07.250108 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:07.279557 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62761729-ae9c-438f-b55b-59b46d8061c6 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:07.300079 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:07.313803 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97641f2-9187-4b1b-9140-f3543b5a4a10 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:07.316887 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54693a3d-6ceb-46d0-b79f-b4af791afe9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:07.316887 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54693a3d-6ceb-46d0-b79f-b4af791afe9f None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:07.317896 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54693a3d-6ceb-46d0-b79f-b4af791afe9f None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:07.318273 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54693a3d-6ceb-46d0-b79f-b4af791afe9f None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:07.319102 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54693a3d-6ceb-46d0-b79f-b4af791afe9f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:07.319535 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54693a3d-6ceb-46d0-b79f-b4af791afe9f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:07.321002 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54693a3d-6ceb-46d0-b79f-b4af791afe9f None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:07.330122 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 401/793] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Aug 6 01:21:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:07.336745 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54693a3d-6ceb-46d0-b79f-b4af791afe9f None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:07.359302 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e44535-d8d1-4256-83cf-5c0f7681ec77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:07.359933 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e44535-d8d1-4256-83cf-5c0f7681ec77 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:07.360216 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e44535-d8d1-4256-83cf-5c0f7681ec77 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:07.360490 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e44535-d8d1-4256-83cf-5c0f7681ec77 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:07.361356 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e44535-d8d1-4256-83cf-5c0f7681ec77 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:07.361664 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e44535-d8d1-4256-83cf-5c0f7681ec77 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:07.363138 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e44535-d8d1-4256-83cf-5c0f7681ec77 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:07.382424 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 393/794] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Aug 6 01:21:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:07.392894 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e44535-d8d1-4256-83cf-5c0f7681ec77 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:07.506008 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54693a3d-6ceb-46d0-b79f-b4af791afe9f None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:07.571792 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396c4bbf-26a5-4de8-ac9f-5eb93e57f64a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:07.572435 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396c4bbf-26a5-4de8-ac9f-5eb93e57f64a None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:07.572720 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396c4bbf-26a5-4de8-ac9f-5eb93e57f64a None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:07.573593 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396c4bbf-26a5-4de8-ac9f-5eb93e57f64a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:07.580220 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e44535-d8d1-4256-83cf-5c0f7681ec77 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:07.582593 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:07.606194 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396c4bbf-26a5-4de8-ac9f-5eb93e57f64a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:07.607656 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396c4bbf-26a5-4de8-ac9f-5eb93e57f64a None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:07.625722 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d3c535-66f5-43cd-a5d7-6c9972089c09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:07.626376 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d3c535-66f5-43cd-a5d7-6c9972089c09 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:07.626664 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d3c535-66f5-43cd-a5d7-6c9972089c09 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:07.626947 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d3c535-66f5-43cd-a5d7-6c9972089c09 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:07.634160 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d3c535-66f5-43cd-a5d7-6c9972089c09 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:07.636514 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d3c535-66f5-43cd-a5d7-6c9972089c09 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:07.643431 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-396c4bbf-26a5-4de8-ac9f-5eb93e57f64a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:21:07.643431 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:07.643431 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:21:07.643431 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:21:07.643431 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:21:07.643431 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:21:07.643431 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:21:07.643431 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:21:07.643431 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:21:07.643431 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:21:07.643431 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:21:07.643431 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:21:07.643431 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:21:07.643431 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:21:07.643431 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:21:07.644326 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:21:07.644326 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:21:07.644326 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:21:07.644326 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:21:07.644326 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:21:07.644326 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:21:07.644326 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:21:07.644326 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:21:07.644326 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:21:07.644326 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:21:07.644326 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:21:07.644326 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:21:07.644326 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:21:07.644326 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:21:07.644326 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:21:07.644326 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:21:07.644326 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:21:07.645269 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:21:07.645269 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:21:07.645269 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:21:07.645269 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:21:07.648222 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-63d3c535-66f5-43cd-a5d7-6c9972089c09 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:21:07.648222 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:07.648222 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:21:07.648222 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:21:07.648222 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:21:07.648222 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:21:07.648222 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:21:07.648222 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:21:07.648222 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:21:07.648222 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:21:07.648222 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:21:07.648222 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:21:07.648222 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 01:21:07.648222 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 01:21:07.648222 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 01:21:07.650209 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 01:21:07.650209 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:21:07.650209 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:21:07.650209 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:21:07.650209 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:21:07.650209 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:21:07.650209 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:21:07.650209 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:21:07.650209 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:21:07.650209 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:21:07.650209 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:21:07.650209 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:21:07.650209 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:21:07.650209 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:21:07.650209 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:21:07.650209 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:21:07.650209 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:21:07.652911 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 01:21:07.652911 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 01:21:07.652911 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 01:21:07.652911 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 01:21:07.652911 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 01:21:07.652911 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 01:21:07.652911 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:21:07.652911 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:21:07.652911 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 394/795] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Aug 6 01:21:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:21:07.654075 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 402/796] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:21:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:21:07.657130 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d3c535-66f5-43cd-a5d7-6c9972089c09 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:07.657337 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396c4bbf-26a5-4de8-ac9f-5eb93e57f64a None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:07.668019 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:07.808953 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d3c535-66f5-43cd-a5d7-6c9972089c09 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:07.882168 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396c4bbf-26a5-4de8-ac9f-5eb93e57f64a None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:07.897747 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569b4157-1897-4037-b41e-e81ca6903988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:07.898701 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569b4157-1897-4037-b41e-e81ca6903988 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:07.899186 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569b4157-1897-4037-b41e-e81ca6903988 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:07.899698 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569b4157-1897-4037-b41e-e81ca6903988 None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:07.900676 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569b4157-1897-4037-b41e-e81ca6903988 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:07.901963 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569b4157-1897-4037-b41e-e81ca6903988 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:07.903737 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569b4157-1897-4037-b41e-e81ca6903988 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:07.920099 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f24fa81-6550-4cc9-9553-35b2df51d3d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:07.920717 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f24fa81-6550-4cc9-9553-35b2df51d3d3 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:07.920996 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f24fa81-6550-4cc9-9553-35b2df51d3d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:07.921298 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f24fa81-6550-4cc9-9553-35b2df51d3d3 None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:07.921991 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f24fa81-6550-4cc9-9553-35b2df51d3d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:07.922294 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f24fa81-6550-4cc9-9553-35b2df51d3d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:07.929071 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f24fa81-6550-4cc9-9553-35b2df51d3d3 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:07.937896 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 403/797] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Aug 6 01:21:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:07.938398 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 395/798] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Aug 6 01:21:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:21:07.946048 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569b4157-1897-4037-b41e-e81ca6903988 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:07.946230 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f24fa81-6550-4cc9-9553-35b2df51d3d3 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:08.014047 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:08.047714 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:08.069606 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f24fa81-6550-4cc9-9553-35b2df51d3d3 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:08.102366 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569b4157-1897-4037-b41e-e81ca6903988 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:08.109566 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304d0db2-6a9b-40f5-b331-77f20b0501f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:08.110413 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304d0db2-6a9b-40f5-b331-77f20b0501f5 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:08.110859 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304d0db2-6a9b-40f5-b331-77f20b0501f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:08.111425 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304d0db2-6a9b-40f5-b331-77f20b0501f5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:08.119373 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304d0db2-6a9b-40f5-b331-77f20b0501f5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:08.121160 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304d0db2-6a9b-40f5-b331-77f20b0501f5 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:08.133807 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-304d0db2-6a9b-40f5-b331-77f20b0501f5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 06 01:21:08.133807 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:08.133807 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:21:08.133807 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:21:08.133807 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:21:08.133807 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:21:08.133807 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:21:08.133807 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:21:08.133807 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:21:08.133807 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:21:08.133807 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:21:08.133807 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:21:08.133807 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 01:21:08.133807 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 01:21:08.133807 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 01:21:08.133807 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 01:21:08.134962 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:21:08.134962 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:21:08.134962 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 01:21:08.134962 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:21:08.134962 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 01:21:08.134962 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:21:08.134962 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:21:08.134962 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:21:08.134962 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:21:08.134962 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:21:08.134962 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:21:08.134962 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:21:08.134962 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:21:08.134962 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:21:08.134962 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:21:08.134962 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:21:08.134962 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:21:08.134962 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:21:08.136654 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:21:08.136654 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:21:08.136654 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 01:21:08.136654 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 01:21:08.136654 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 01:21:08.136654 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 01:21:08.136654 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 01:21:08.136654 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 01:21:08.136654 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 06 01:21:08.136654 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:21:08.139098 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 404/799] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Aug 6 01:21:07 2022] GET /identity/v3/users/swift => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:21:08.140525 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443264d8-62ec-4f8e-b862-94a24aea7f01 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:08.141002 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443264d8-62ec-4f8e-b862-94a24aea7f01 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:08.141751 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443264d8-62ec-4f8e-b862-94a24aea7f01 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:08.143427 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 405/800] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:21:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:21:08.144128 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443264d8-62ec-4f8e-b862-94a24aea7f01 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:08.152628 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfcab67-09c9-4df0-a7c1-aedc6eda2875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:08.153359 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfcab67-09c9-4df0-a7c1-aedc6eda2875 None admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:08.153647 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfcab67-09c9-4df0-a7c1-aedc6eda2875 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:08.153929 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfcab67-09c9-4df0-a7c1-aedc6eda2875 None admin] PATH_INFO: `/v3/projects/d9d68d310bf24449a5fe20d2dc264a58/users/8e86f38590824894a3b960e6774ff8a9/roles/c9ce798acd4b482f8f391c6af97eb7a4` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:08.174237 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfcab67-09c9-4df0-a7c1-aedc6eda2875 None admin] RBAC: Authorizing `identity:create_grant(project_id=d9d68d310bf24449a5fe20d2dc264a58, user_id=8e86f38590824894a3b960e6774ff8a9, role_id=c9ce798acd4b482f8f391c6af97eb7a4)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:08.176350 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfcab67-09c9-4df0-a7c1-aedc6eda2875 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:08.197130 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 396/801] 10.222.0.26 () {58 vars in 1439 bytes} [Sat Aug 6 01:21:07 2022] PUT /identity/v3/projects/d9d68d310bf24449a5fe20d2dc264a58/users/8e86f38590824894a3b960e6774ff8a9/roles/c9ce798acd4b482f8f391c6af97eb7a4 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:21:08.199692 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f588d1-07d9-41ec-b166-5b091752f4b2 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:08.199989 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f588d1-07d9-41ec-b166-5b091752f4b2 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:08.200270 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f588d1-07d9-41ec-b166-5b091752f4b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:08.223455 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-45f588d1-07d9-41ec-b166-5b091752f4b2 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:21:08.224781 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-45f588d1-07d9-41ec-b166-5b091752f4b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:21:08.306308 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 397/802] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:21:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:21:08.326044 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443264d8-62ec-4f8e-b862-94a24aea7f01 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:08.336411 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:08.341698 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f588d1-07d9-41ec-b166-5b091752f4b2 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:08.418840 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85ac99b-4cdb-469c-ada7-a3b450d981bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:08.419479 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85ac99b-4cdb-469c-ada7-a3b450d981bf None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:08.419973 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85ac99b-4cdb-469c-ada7-a3b450d981bf None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:08.420252 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85ac99b-4cdb-469c-ada7-a3b450d981bf None admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:08.421868 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85ac99b-4cdb-469c-ada7-a3b450d981bf None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:08.422186 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85ac99b-4cdb-469c-ada7-a3b450d981bf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:08.423690 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85ac99b-4cdb-469c-ada7-a3b450d981bf None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:08.442582 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:08.457335 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 406/803] 10.222.0.26 () {56 vars in 1135 bytes} [Sat Aug 6 01:21:08 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:08.458825 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa462cd-2a0c-4be9-be35-5dc69963cb02 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:08.459108 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa462cd-2a0c-4be9-be35-5dc69963cb02 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:08.459377 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa462cd-2a0c-4be9-be35-5dc69963cb02 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:08.465164 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 407/804] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:21:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:21:08.466144 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa462cd-2a0c-4be9-be35-5dc69963cb02 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:08.515671 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f588d1-07d9-41ec-b166-5b091752f4b2 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:08.554544 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d55059e-d51f-46b7-9fad-69ae5a959ccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:08.555537 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d55059e-d51f-46b7-9fad-69ae5a959ccd None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:08.556456 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d55059e-d51f-46b7-9fad-69ae5a959ccd None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:08.557039 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d55059e-d51f-46b7-9fad-69ae5a959ccd None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:08.558938 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d55059e-d51f-46b7-9fad-69ae5a959ccd None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:08.561150 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d55059e-d51f-46b7-9fad-69ae5a959ccd None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:08.574177 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-1d55059e-d51f-46b7-9fad-69ae5a959ccd None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 06 01:21:08.574177 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:08.574177 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 01:21:08.574177 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 01:21:08.574177 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 01:21:08.574177 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 01:21:08.574177 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 01:21:08.574177 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 01:21:08.574177 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 06 01:21:08.574177 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:21:08.574177 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 01:21:08.574177 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:21:08.574177 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:08.574177 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:21:08.574177 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:21:08.574177 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:21:08.575135 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:21:08.575135 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:21:08.575135 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:21:08.575135 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:21:08.575135 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:21:08.575135 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:21:08.575135 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:21:08.575135 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 01:21:08.575135 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 01:21:08.575135 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 01:21:08.575135 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 01:21:08.575135 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:21:08.575135 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:21:08.575135 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:21:08.575135 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:21:08.575135 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:21:08.575962 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:21:08.575962 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:21:08.575962 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:21:08.575962 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:21:08.575962 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:21:08.575962 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:21:08.575962 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:21:08.575962 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:21:08.575962 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:21:08.575962 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:21:08.575962 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:21:08.575962 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 01:21:08.575962 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 01:21:08.575962 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 06 01:21:08.575962 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:21:08.579220 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 398/805] 10.222.0.26 () {56 vars in 1129 bytes} [Sat Aug 6 01:21:08 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:21:08.580903 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a10588b-b750-4f37-9d97-2f8ae95d7850 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:08.582224 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a10588b-b750-4f37-9d97-2f8ae95d7850 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:08.582946 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a10588b-b750-4f37-9d97-2f8ae95d7850 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:08.585783 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa462cd-2a0c-4be9-be35-5dc69963cb02 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:08.606268 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-5a10588b-b750-4f37-9d97-2f8ae95d7850 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:21:08.608620 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-5a10588b-b750-4f37-9d97-2f8ae95d7850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:21:08.630561 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1390b9a3-34e0-4db9-8e55-11615fdc4735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:08.631350 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1390b9a3-34e0-4db9-8e55-11615fdc4735 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:08.631727 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1390b9a3-34e0-4db9-8e55-11615fdc4735 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:08.632100 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1390b9a3-34e0-4db9-8e55-11615fdc4735 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:08.643226 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1390b9a3-34e0-4db9-8e55-11615fdc4735 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:08.644930 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1390b9a3-34e0-4db9-8e55-11615fdc4735 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:08.661395 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-1390b9a3-34e0-4db9-8e55-11615fdc4735 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:21:08.661395 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:08.661395 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:21:08.661395 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:21:08.661395 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:21:08.661395 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:21:08.661395 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:21:08.661395 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:21:08.661395 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:21:08.661395 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:21:08.661395 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:21:08.661395 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:21:08.661395 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:21:08.661395 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:21:08.661395 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:21:08.662485 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:21:08.662485 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:21:08.662485 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:21:08.662485 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:21:08.662485 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:21:08.662485 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:21:08.662485 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:21:08.662485 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:21:08.662485 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:21:08.662485 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:21:08.662485 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:21:08.662485 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:21:08.662485 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:21:08.662485 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:21:08.662485 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:21:08.662485 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:21:08.662485 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:21:08.663593 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:21:08.663593 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:21:08.663593 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:21:08.663593 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:21:08.664964 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 408/806] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:21:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:21:08.665847 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1390b9a3-34e0-4db9-8e55-11615fdc4735 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:08.687324 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:08.705765 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 399/807] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:21:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:21:08.706968 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a10588b-b750-4f37-9d97-2f8ae95d7850 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:08.760071 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:08.803405 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1390b9a3-34e0-4db9-8e55-11615fdc4735 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:08.842053 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8627f8d8-7a2c-4ef4-bb9e-e5e3918e3486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:08.842734 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8627f8d8-7a2c-4ef4-bb9e-e5e3918e3486 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:08.843110 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8627f8d8-7a2c-4ef4-bb9e-e5e3918e3486 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:08.843479 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8627f8d8-7a2c-4ef4-bb9e-e5e3918e3486 None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:08.844264 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8627f8d8-7a2c-4ef4-bb9e-e5e3918e3486 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:08.844814 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a10588b-b750-4f37-9d97-2f8ae95d7850 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:08.844947 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8627f8d8-7a2c-4ef4-bb9e-e5e3918e3486 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:08.846520 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8627f8d8-7a2c-4ef4-bb9e-e5e3918e3486 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:08.852945 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 409/808] 10.222.0.26 () {56 vars in 1124 bytes} [Sat Aug 6 01:21:08 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:08.854005 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8627f8d8-7a2c-4ef4-bb9e-e5e3918e3486 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:08.888005 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca40ea99-e5cb-45c7-b9a0-9049e346fc4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:08.888808 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca40ea99-e5cb-45c7-b9a0-9049e346fc4c None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:08.889235 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca40ea99-e5cb-45c7-b9a0-9049e346fc4c None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:08.889739 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca40ea99-e5cb-45c7-b9a0-9049e346fc4c None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:08.890402 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca40ea99-e5cb-45c7-b9a0-9049e346fc4c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:08.890804 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca40ea99-e5cb-45c7-b9a0-9049e346fc4c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:08.892272 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca40ea99-e5cb-45c7-b9a0-9049e346fc4c None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:08.906343 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 400/809] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Aug 6 01:21:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:08.907531 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca40ea99-e5cb-45c7-b9a0-9049e346fc4c None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:08.961291 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:08.982647 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8627f8d8-7a2c-4ef4-bb9e-e5e3918e3486 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:09.029721 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8d07a0-3c6e-4a82-8b5d-e69ad4782b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:09.030377 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8d07a0-3c6e-4a82-8b5d-e69ad4782b3c None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:09.030661 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8d07a0-3c6e-4a82-8b5d-e69ad4782b3c None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:09.030932 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8d07a0-3c6e-4a82-8b5d-e69ad4782b3c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:09.038400 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca40ea99-e5cb-45c7-b9a0-9049e346fc4c None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:09.040416 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:09.043589 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8d07a0-3c6e-4a82-8b5d-e69ad4782b3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:21:09.049026 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-8f8d07a0-3c6e-4a82-8b5d-e69ad4782b3c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:21:09.049026 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:09.049026 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:21:09.049026 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:21:09.049026 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:21:09.049026 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:21:09.049026 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:21:09.049026 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:21:09.049026 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:21:09.049026 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:21:09.049026 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:21:09.049026 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:21:09.049026 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:21:09.049026 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:21:09.049026 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:21:09.049026 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:21:09.050048 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:21:09.050048 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:21:09.050048 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:21:09.050048 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:21:09.050048 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:21:09.050048 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:21:09.050048 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:21:09.050048 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:21:09.050048 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:21:09.050048 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:21:09.050048 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:21:09.050048 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:21:09.050048 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:21:09.050048 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:21:09.050048 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:21:09.050048 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:21:09.050907 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:21:09.050907 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:21:09.050907 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:21:09.050907 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:21:09.050907 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:21:09.050907 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:21:09.050907 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:21:09.050907 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:21:09.050907 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:21:09.050907 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:21:09.050907 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:21:09.050907 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:21:09.055177 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 410/810] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Aug 6 01:21:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:21:09.056116 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8d07a0-3c6e-4a82-8b5d-e69ad4782b3c None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:09.086290 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49c8f62-5c60-424e-8d1d-d3f7bfe84524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:09.087306 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49c8f62-5c60-424e-8d1d-d3f7bfe84524 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:09.087849 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49c8f62-5c60-424e-8d1d-d3f7bfe84524 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:09.088440 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49c8f62-5c60-424e-8d1d-d3f7bfe84524 None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:09.089693 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49c8f62-5c60-424e-8d1d-d3f7bfe84524 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:09.090217 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49c8f62-5c60-424e-8d1d-d3f7bfe84524 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:09.091806 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49c8f62-5c60-424e-8d1d-d3f7bfe84524 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:09.097967 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 401/811] 10.222.0.26 () {56 vars in 1124 bytes} [Sat Aug 6 01:21:08 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:09.099135 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49c8f62-5c60-424e-8d1d-d3f7bfe84524 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:09.163317 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8d07a0-3c6e-4a82-8b5d-e69ad4782b3c None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:09.196246 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db56e51-51f5-40c0-9998-b3b394046f7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:09.196915 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db56e51-51f5-40c0-9998-b3b394046f7f None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:09.197193 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db56e51-51f5-40c0-9998-b3b394046f7f None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:09.197470 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db56e51-51f5-40c0-9998-b3b394046f7f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:09.204880 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db56e51-51f5-40c0-9998-b3b394046f7f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:09.206439 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db56e51-51f5-40c0-9998-b3b394046f7f None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:09.219397 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-8db56e51-51f5-40c0-9998-b3b394046f7f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:21:09.219397 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:09.219397 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:21:09.219397 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:21:09.219397 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:21:09.219397 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:21:09.219397 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:21:09.219397 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:21:09.219397 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:21:09.219397 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:21:09.219397 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:21:09.219397 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:21:09.219397 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 01:21:09.219397 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 01:21:09.219397 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 01:21:09.220327 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 01:21:09.220327 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:21:09.220327 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:21:09.220327 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:21:09.220327 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:21:09.220327 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:21:09.220327 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:21:09.220327 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:21:09.220327 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:21:09.220327 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:21:09.220327 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:21:09.220327 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:21:09.220327 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:21:09.220327 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:21:09.220327 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:21:09.220327 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:21:09.220327 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:21:09.221842 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 01:21:09.221842 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 01:21:09.221842 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 01:21:09.221842 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 01:21:09.221842 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 01:21:09.221842 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 01:21:09.221842 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:21:09.221842 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:21:09.221842 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 411/812] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Aug 6 01:21:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:21:09.221842 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db56e51-51f5-40c0-9998-b3b394046f7f None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:09.224782 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:09.230915 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49c8f62-5c60-424e-8d1d-d3f7bfe84524 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:09.269727 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81ebad6-054f-4f58-b335-c73b99aec99b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:09.270705 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81ebad6-054f-4f58-b335-c73b99aec99b None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:09.271228 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81ebad6-054f-4f58-b335-c73b99aec99b None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:09.271800 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81ebad6-054f-4f58-b335-c73b99aec99b None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:09.272701 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81ebad6-054f-4f58-b335-c73b99aec99b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:09.273255 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81ebad6-054f-4f58-b335-c73b99aec99b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:09.274905 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81ebad6-054f-4f58-b335-c73b99aec99b None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:09.289835 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 402/813] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Aug 6 01:21:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:09.290975 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81ebad6-054f-4f58-b335-c73b99aec99b None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:09.325325 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:09.328737 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db56e51-51f5-40c0-9998-b3b394046f7f None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:09.365125 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1474ed9-afc1-4e68-9fd6-a11981d3683e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:09.365681 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1474ed9-afc1-4e68-9fd6-a11981d3683e None admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:09.366036 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1474ed9-afc1-4e68-9fd6-a11981d3683e None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:09.366321 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1474ed9-afc1-4e68-9fd6-a11981d3683e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:09.367005 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1474ed9-afc1-4e68-9fd6-a11981d3683e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:09.368363 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1474ed9-afc1-4e68-9fd6-a11981d3683e None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:09.389993 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 412/814] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:21:09 2022] POST /identity/v3/endpoints => generated 346 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:21:09.396373 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1474ed9-afc1-4e68-9fd6-a11981d3683e None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:09.421142 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81ebad6-054f-4f58-b335-c73b99aec99b None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:09.468261 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb88f194-b82f-4462-9994-aedccf900037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:09.469101 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb88f194-b82f-4462-9994-aedccf900037 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:09.469457 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb88f194-b82f-4462-9994-aedccf900037 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:09.469804 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb88f194-b82f-4462-9994-aedccf900037 None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:09.470807 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb88f194-b82f-4462-9994-aedccf900037 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:09.471185 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb88f194-b82f-4462-9994-aedccf900037 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:09.472798 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb88f194-b82f-4462-9994-aedccf900037 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:09.477991 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:09.493015 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 403/815] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Aug 6 01:21:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:09.494433 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d09f401-b1f9-4073-b60e-9856c1191d4a None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:09.494768 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d09f401-b1f9-4073-b60e-9856c1191d4a None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:09.495053 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d09f401-b1f9-4073-b60e-9856c1191d4a None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:09.497099 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 404/816] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:21:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:21:09.506049 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d09f401-b1f9-4073-b60e-9856c1191d4a None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:09.534733 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1474ed9-afc1-4e68-9fd6-a11981d3683e None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:09.573533 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64f55d7-e061-4c24-8825-649524ddfce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:09.574367 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64f55d7-e061-4c24-8825-649524ddfce6 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:09.574634 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64f55d7-e061-4c24-8825-649524ddfce6 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:09.574910 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64f55d7-e061-4c24-8825-649524ddfce6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:09.586189 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64f55d7-e061-4c24-8825-649524ddfce6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:09.587631 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64f55d7-e061-4c24-8825-649524ddfce6 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:09.603842 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-f64f55d7-e061-4c24-8825-649524ddfce6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:21:09.603842 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:09.603842 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:21:09.603842 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:21:09.603842 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:21:09.603842 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:21:09.603842 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:21:09.603842 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:21:09.603842 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:21:09.603842 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:21:09.603842 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:21:09.603842 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:21:09.603842 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:21:09.603842 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:21:09.603842 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:21:09.605442 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:21:09.605442 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:21:09.605442 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:21:09.605442 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:21:09.605442 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:21:09.605442 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:21:09.605442 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:21:09.605442 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:21:09.605442 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:21:09.605442 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:21:09.605442 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:21:09.605442 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:21:09.605442 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:21:09.605442 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:21:09.605442 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:21:09.605442 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:21:09.605442 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:21:09.606443 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:21:09.606443 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:21:09.606443 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:21:09.606443 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:21:09.606443 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 413/817] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:21:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:21:09.607591 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be0d6aa-1d83-4f3d-80e7-d1bb1f6719ef None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:09.607925 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be0d6aa-1d83-4f3d-80e7-d1bb1f6719ef None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:09.608211 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be0d6aa-1d83-4f3d-80e7-d1bb1f6719ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:09.614652 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:09.627785 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-2be0d6aa-1d83-4f3d-80e7-d1bb1f6719ef None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:21:09.628999 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-2be0d6aa-1d83-4f3d-80e7-d1bb1f6719ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:21:09.636194 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d09f401-b1f9-4073-b60e-9856c1191d4a None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:09.675765 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca8e406-e614-4a16-8bd5-706db5fde79a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:09.676493 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca8e406-e614-4a16-8bd5-706db5fde79a None admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:09.676778 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca8e406-e614-4a16-8bd5-706db5fde79a None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:09.677093 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca8e406-e614-4a16-8bd5-706db5fde79a None admin] PATH_INFO: `/v3/projects/d9d68d310bf24449a5fe20d2dc264a58/users/2b23f4478f49471fba6f43e6e4be99da/roles/4559b9a0837249f895a13c70096f8d38` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:09.679116 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca8e406-e614-4a16-8bd5-706db5fde79a None admin] RBAC: Authorizing `identity:create_grant(project_id=d9d68d310bf24449a5fe20d2dc264a58, user_id=2b23f4478f49471fba6f43e6e4be99da, role_id=4559b9a0837249f895a13c70096f8d38)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:09.680546 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca8e406-e614-4a16-8bd5-706db5fde79a None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:09.696093 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 405/818] 10.222.0.26 () {58 vars in 1439 bytes} [Sat Aug 6 01:21:09 2022] PUT /identity/v3/projects/d9d68d310bf24449a5fe20d2dc264a58/users/2b23f4478f49471fba6f43e6e4be99da/roles/4559b9a0837249f895a13c70096f8d38 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:21:09.696955 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca8e406-e614-4a16-8bd5-706db5fde79a None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:09.710286 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 414/819] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:21:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:21:09.734316 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be0d6aa-1d83-4f3d-80e7-d1bb1f6719ef None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:09.817030 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca8e406-e614-4a16-8bd5-706db5fde79a None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:09.852065 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:09.859035 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2333c5b-8f4f-4ce2-90dc-5cd2f890a13f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:09.859908 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2333c5b-8f4f-4ce2-90dc-5cd2f890a13f None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:09.860367 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2333c5b-8f4f-4ce2-90dc-5cd2f890a13f None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:09.860821 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2333c5b-8f4f-4ce2-90dc-5cd2f890a13f None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:09.861749 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2333c5b-8f4f-4ce2-90dc-5cd2f890a13f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:09.862192 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2333c5b-8f4f-4ce2-90dc-5cd2f890a13f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:09.863750 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2333c5b-8f4f-4ce2-90dc-5cd2f890a13f None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:09.879078 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 406/820] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Aug 6 01:21:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:09.891992 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2333c5b-8f4f-4ce2-90dc-5cd2f890a13f None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:09.903066 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:09.961367 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be0d6aa-1d83-4f3d-80e7-d1bb1f6719ef None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:10.009331 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5e9a51-ec60-425c-b1d9-0a9aee1ca085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:10.010544 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5e9a51-ec60-425c-b1d9-0a9aee1ca085 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:10.011110 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5e9a51-ec60-425c-b1d9-0a9aee1ca085 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:10.011936 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5e9a51-ec60-425c-b1d9-0a9aee1ca085 None admin] PATH_INFO: `/v3/services/network` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:10.012686 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5e9a51-ec60-425c-b1d9-0a9aee1ca085 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:10.014084 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2333c5b-8f4f-4ce2-90dc-5cd2f890a13f None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:10.014577 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5e9a51-ec60-425c-b1d9-0a9aee1ca085 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:10.027138 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-1e5e9a51-ec60-425c-b1d9-0a9aee1ca085 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 06 01:21:10.027138 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:10.027138 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 01:21:10.027138 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 01:21:10.027138 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 01:21:10.027138 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 01:21:10.027138 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 01:21:10.027138 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 01:21:10.027138 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 06 01:21:10.027138 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:21:10.027138 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 01:21:10.027138 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:21:10.027138 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:10.027138 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:21:10.027138 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:21:10.027138 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:21:10.027138 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:21:10.028437 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:21:10.028437 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:21:10.028437 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:21:10.028437 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:21:10.028437 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:21:10.028437 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:21:10.028437 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 01:21:10.028437 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 01:21:10.028437 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 01:21:10.028437 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 01:21:10.028437 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:21:10.028437 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:21:10.028437 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:21:10.028437 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:21:10.028437 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:21:10.028437 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:21:10.029493 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:21:10.029493 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:21:10.029493 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:21:10.029493 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:21:10.029493 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:21:10.029493 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:21:10.029493 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:21:10.029493 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:21:10.029493 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:21:10.029493 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:21:10.029493 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 01:21:10.029493 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 01:21:10.029493 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 06 01:21:10.029493 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:21:10.033120 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 415/821] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Aug 6 01:21:09 2022] GET /identity/v3/services/network => generated 88 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:21:10.039295 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5e9a51-ec60-425c-b1d9-0a9aee1ca085 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:10.096510 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decfe849-5c29-4563-be98-173a78bce42a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:10.097566 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decfe849-5c29-4563-be98-173a78bce42a None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:10.098046 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decfe849-5c29-4563-be98-173a78bce42a None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:10.098534 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decfe849-5c29-4563-be98-173a78bce42a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:10.117867 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decfe849-5c29-4563-be98-173a78bce42a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:10.119647 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decfe849-5c29-4563-be98-173a78bce42a None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:10.139077 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-decfe849-5c29-4563-be98-173a78bce42a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 06 01:21:10.139077 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:10.139077 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:21:10.139077 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:21:10.139077 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:21:10.139077 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:21:10.139077 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:21:10.139077 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:21:10.139077 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:21:10.139077 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:21:10.139077 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:21:10.139077 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:21:10.139077 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 01:21:10.139077 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 01:21:10.139077 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 01:21:10.139077 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 01:21:10.140033 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:21:10.140033 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:21:10.140033 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 01:21:10.140033 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:21:10.140033 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 01:21:10.140033 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:21:10.140033 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:21:10.140033 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:21:10.140033 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:21:10.140033 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:21:10.140033 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:21:10.140033 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:21:10.140033 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:21:10.140033 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:21:10.140033 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:21:10.140033 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:21:10.140033 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:21:10.140033 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:21:10.140986 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:21:10.140986 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:21:10.140986 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 01:21:10.140986 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 01:21:10.140986 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 01:21:10.140986 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 01:21:10.140986 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 01:21:10.140986 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 01:21:10.140986 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 06 01:21:10.140986 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:21:10.152121 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 407/822] 10.222.0.26 () {56 vars in 1093 bytes} [Sat Aug 6 01:21:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:21:10.155381 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5e9a51-ec60-425c-b1d9-0a9aee1ca085 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:10.158233 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decfe849-5c29-4563-be98-173a78bce42a None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:10.158353 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:10.193270 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a431087c-d455-4c01-bd6e-b927771f0526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:10.193982 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a431087c-d455-4c01-bd6e-b927771f0526 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:10.194407 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a431087c-d455-4c01-bd6e-b927771f0526 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:10.194755 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a431087c-d455-4c01-bd6e-b927771f0526 None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:10.195525 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a431087c-d455-4c01-bd6e-b927771f0526 None admin] RBAC: Adding query filter params (name=network) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:10.195899 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a431087c-d455-4c01-bd6e-b927771f0526 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:10.197398 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a431087c-d455-4c01-bd6e-b927771f0526 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:10.203129 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 416/823] 10.222.0.26 () {56 vars in 1112 bytes} [Sat Aug 6 01:21:10 2022] GET /identity/v3/services?name=network => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:10.208823 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a431087c-d455-4c01-bd6e-b927771f0526 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:10.265457 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:10.314612 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a431087c-d455-4c01-bd6e-b927771f0526 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:10.316261 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decfe849-5c29-4563-be98-173a78bce42a None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:10.350917 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be498f81-4160-4b99-8098-5f740374eff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:10.351998 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be498f81-4160-4b99-8098-5f740374eff6 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:10.352351 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be498f81-4160-4b99-8098-5f740374eff6 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:10.352703 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be498f81-4160-4b99-8098-5f740374eff6 None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:10.353490 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be498f81-4160-4b99-8098-5f740374eff6 None admin] RBAC: Adding query filter params (type=network) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:10.353865 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be498f81-4160-4b99-8098-5f740374eff6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:10.356036 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be498f81-4160-4b99-8098-5f740374eff6 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:10.361649 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 417/824] 10.222.0.26 () {56 vars in 1112 bytes} [Sat Aug 6 01:21:10 2022] GET /identity/v3/services?type=network => generated 350 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:21:10.367176 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be498f81-4160-4b99-8098-5f740374eff6 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:10.376761 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adab33a-b473-4f9a-875b-f48886ab1937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:10.377436 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adab33a-b473-4f9a-875b-f48886ab1937 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:10.377722 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adab33a-b473-4f9a-875b-f48886ab1937 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:10.378001 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adab33a-b473-4f9a-875b-f48886ab1937 None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:10.378917 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adab33a-b473-4f9a-875b-f48886ab1937 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:10.379232 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adab33a-b473-4f9a-875b-f48886ab1937 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:10.380791 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adab33a-b473-4f9a-875b-f48886ab1937 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:10.409340 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:10.420880 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 408/825] 10.222.0.26 () {56 vars in 1133 bytes} [Sat Aug 6 01:21:10 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:21:10.422478 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adab33a-b473-4f9a-875b-f48886ab1937 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:10.522664 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be498f81-4160-4b99-8098-5f740374eff6 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:10.563857 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b52fc61-662b-44ff-8f0e-e684126526b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:10.564533 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b52fc61-662b-44ff-8f0e-e684126526b4 None admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:10.564822 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b52fc61-662b-44ff-8f0e-e684126526b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:10.565119 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b52fc61-662b-44ff-8f0e-e684126526b4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:10.565834 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b52fc61-662b-44ff-8f0e-e684126526b4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:10.567241 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b52fc61-662b-44ff-8f0e-e684126526b4 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:10.573946 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adab33a-b473-4f9a-875b-f48886ab1937 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:10.591744 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 418/826] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:21:10 2022] POST /identity/v3/endpoints => generated 337 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:21:10.593273 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a090a93-c66c-477d-9898-6163393284ea None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:10.593716 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a090a93-c66c-477d-9898-6163393284ea None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:10.594107 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a090a93-c66c-477d-9898-6163393284ea None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:10.595915 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 419/827] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:21:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:21:10.601224 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f159a6f3-dc2c-40cd-9022-bf42cd8cf904 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:10.601552 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f159a6f3-dc2c-40cd-9022-bf42cd8cf904 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:10.601807 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f159a6f3-dc2c-40cd-9022-bf42cd8cf904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:10.605093 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:10.625935 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159cc40a-7fe8-461c-81b3-a2733da8a3a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:10.627457 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159cc40a-7fe8-461c-81b3-a2733da8a3a6 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:10.627936 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159cc40a-7fe8-461c-81b3-a2733da8a3a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:10.628421 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159cc40a-7fe8-461c-81b3-a2733da8a3a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:10.654344 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-f159a6f3-dc2c-40cd-9022-bf42cd8cf904 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:21:10.655607 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-f159a6f3-dc2c-40cd-9022-bf42cd8cf904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:21:10.740543 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159cc40a-7fe8-461c-81b3-a2733da8a3a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:10.747503 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159cc40a-7fe8-461c-81b3-a2733da8a3a6 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:10.807780 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 420/828] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:21:10 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 06 01:21:10.829181 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f159a6f3-dc2c-40cd-9022-bf42cd8cf904 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:10.851815 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:10.906267 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 409/829] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Aug 6 01:21:10 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 06 01:21:10.909160 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159cc40a-7fe8-461c-81b3-a2733da8a3a6 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:10.934674 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:10.958215 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f159a6f3-dc2c-40cd-9022-bf42cd8cf904 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:11.002713 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ea6e60-f4fb-4384-8590-d2431e518d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:11.003458 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ea6e60-f4fb-4384-8590-d2431e518d00 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:11.003640 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ea6e60-f4fb-4384-8590-d2431e518d00 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:11.003915 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ea6e60-f4fb-4384-8590-d2431e518d00 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:11.022329 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ea6e60-f4fb-4384-8590-d2431e518d00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:21:11.022540 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-a3ea6e60-f4fb-4384-8590-d2431e518d00 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 06 01:21:11.022540 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:11.022540 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:21:11.022540 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:21:11.022540 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:21:11.022540 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:21:11.022540 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:21:11.022540 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:21:11.022540 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:21:11.022540 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:21:11.022540 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:21:11.022540 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:21:11.022540 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:21:11.022540 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:21:11.022540 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:21:11.022540 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:21:11.023655 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:21:11.023655 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:21:11.023655 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:21:11.023655 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:21:11.023655 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:21:11.023655 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:21:11.023655 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:21:11.023655 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:21:11.023655 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:21:11.023655 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:21:11.023655 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:21:11.023655 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:21:11.023655 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:21:11.023655 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:21:11.023655 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:21:11.023655 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:21:11.025019 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:21:11.025019 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:21:11.025019 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:21:11.025019 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:21:11.025019 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:21:11.025019 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:21:11.025019 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:21:11.025019 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:21:11.025019 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:21:11.025019 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:21:11.025019 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 06 01:21:11.025019 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:21:11.025019 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 421/830] 10.222.0.26 () {56 vars in 1102 bytes} [Sat Aug 6 01:21:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:21:11.031472 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ea6e60-f4fb-4384-8590-d2431e518d00 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:11.049930 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159cc40a-7fe8-461c-81b3-a2733da8a3a6 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:11.096213 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb6a368-6556-4032-9fb5-f050791611af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:11.097295 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb6a368-6556-4032-9fb5-f050791611af None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:11.097922 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb6a368-6556-4032-9fb5-f050791611af None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:11.098411 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb6a368-6556-4032-9fb5-f050791611af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:11.111733 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb6a368-6556-4032-9fb5-f050791611af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:11.113617 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb6a368-6556-4032-9fb5-f050791611af None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:11.130267 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-acb6a368-6556-4032-9fb5-f050791611af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:21:11.130267 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:11.130267 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:21:11.130267 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:21:11.130267 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:21:11.130267 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:21:11.130267 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:21:11.130267 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:21:11.130267 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:21:11.130267 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:21:11.130267 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:21:11.130267 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:21:11.130267 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:21:11.130267 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:21:11.130267 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:21:11.131189 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:21:11.131189 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:21:11.131189 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:21:11.131189 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:21:11.131189 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:21:11.131189 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:21:11.131189 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:21:11.131189 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:21:11.131189 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:21:11.131189 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:21:11.131189 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:21:11.131189 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:21:11.131189 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:21:11.131189 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:21:11.131189 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:21:11.131189 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:21:11.131189 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:21:11.132149 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:21:11.132149 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:21:11.132149 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:21:11.132149 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:21:11.139404 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 410/831] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:21:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:21:11.140853 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:11.142137 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb6a368-6556-4032-9fb5-f050791611af None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:11.167621 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ea6e60-f4fb-4384-8590-d2431e518d00 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:11.219750 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259f73b2-0e15-4930-81c3-e0919617b225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:11.220441 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259f73b2-0e15-4930-81c3-e0919617b225 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:11.220728 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259f73b2-0e15-4930-81c3-e0919617b225 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:11.221004 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259f73b2-0e15-4930-81c3-e0919617b225 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:11.222020 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259f73b2-0e15-4930-81c3-e0919617b225 None admin] RBAC: Adding query filter params (name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:11.222329 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259f73b2-0e15-4930-81c3-e0919617b225 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:11.224424 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259f73b2-0e15-4930-81c3-e0919617b225 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:11.232619 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:11.237571 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 422/832] 10.222.0.26 () {56 vars in 1103 bytes} [Sat Aug 6 01:21:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:21:11.246274 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259f73b2-0e15-4930-81c3-e0919617b225 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:11.276115 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb6a368-6556-4032-9fb5-f050791611af None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:11.314835 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab73a2f4-24b5-4215-aaae-343ce232d12e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:11.315802 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab73a2f4-24b5-4215-aaae-343ce232d12e None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:11.316313 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab73a2f4-24b5-4215-aaae-343ce232d12e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:11.316814 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab73a2f4-24b5-4215-aaae-343ce232d12e None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:11.317812 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab73a2f4-24b5-4215-aaae-343ce232d12e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:11.318338 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab73a2f4-24b5-4215-aaae-343ce232d12e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:11.320910 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab73a2f4-24b5-4215-aaae-343ce232d12e None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:11.334495 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 411/833] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Aug 6 01:21:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:11.340156 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab73a2f4-24b5-4215-aaae-343ce232d12e None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:11.382192 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259f73b2-0e15-4930-81c3-e0919617b225 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:11.434744 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca72de55-5e0b-471e-b1b3-1bd3b6eb3847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:11.435981 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca72de55-5e0b-471e-b1b3-1bd3b6eb3847 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:11.436757 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca72de55-5e0b-471e-b1b3-1bd3b6eb3847 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:11.437687 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca72de55-5e0b-471e-b1b3-1bd3b6eb3847 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:11.446619 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab73a2f4-24b5-4215-aaae-343ce232d12e None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:11.451169 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:11.454644 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca72de55-5e0b-471e-b1b3-1bd3b6eb3847 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:11.456987 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca72de55-5e0b-471e-b1b3-1bd3b6eb3847 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:11.478662 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-ca72de55-5e0b-471e-b1b3-1bd3b6eb3847 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:21:11.478662 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:11.478662 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:21:11.478662 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:21:11.478662 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:21:11.478662 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:21:11.478662 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:21:11.478662 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:21:11.478662 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:21:11.478662 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:21:11.478662 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:21:11.478662 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:21:11.478662 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:21:11.478662 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:21:11.478662 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:21:11.479767 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:21:11.479767 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:21:11.479767 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:21:11.479767 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:21:11.479767 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:21:11.479767 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:21:11.479767 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:21:11.479767 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:21:11.479767 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:21:11.479767 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:21:11.479767 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:21:11.479767 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:21:11.479767 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:21:11.479767 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:21:11.479767 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:21:11.479767 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:21:11.479767 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:21:11.480673 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:21:11.480673 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:21:11.480673 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:21:11.480673 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:21:11.481906 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:11.487872 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 423/834] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:21:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:21:11.489773 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca72de55-5e0b-471e-b1b3-1bd3b6eb3847 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:11.495483 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a0951b-eb9f-4ef0-b658-9be38f90d8b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:11.495994 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a0951b-eb9f-4ef0-b658-9be38f90d8b1 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:11.496276 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a0951b-eb9f-4ef0-b658-9be38f90d8b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:11.496555 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a0951b-eb9f-4ef0-b658-9be38f90d8b1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:11.504103 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a0951b-eb9f-4ef0-b658-9be38f90d8b1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:11.505694 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a0951b-eb9f-4ef0-b658-9be38f90d8b1 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:11.518719 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-f1a0951b-eb9f-4ef0-b658-9be38f90d8b1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:21:11.518719 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:11.518719 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:21:11.518719 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:21:11.518719 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:21:11.518719 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:21:11.518719 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:21:11.518719 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:21:11.518719 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:21:11.518719 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:21:11.518719 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:21:11.518719 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:21:11.518719 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 01:21:11.518719 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 01:21:11.518719 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 01:21:11.519624 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 01:21:11.519624 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:21:11.519624 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:21:11.519624 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:21:11.519624 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:21:11.519624 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:21:11.519624 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:21:11.519624 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:21:11.519624 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:21:11.519624 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:21:11.519624 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:21:11.519624 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:21:11.519624 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:21:11.519624 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:21:11.519624 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:21:11.519624 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:21:11.519624 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:21:11.521087 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 01:21:11.521087 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 01:21:11.521087 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 01:21:11.521087 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 01:21:11.521087 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 01:21:11.521087 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 01:21:11.521087 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:21:11.521087 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:21:11.524656 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 412/835] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Aug 6 01:21:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:21:11.525668 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a0951b-eb9f-4ef0-b658-9be38f90d8b1 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:11.626984 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca72de55-5e0b-471e-b1b3-1bd3b6eb3847 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:11.637313 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a0951b-eb9f-4ef0-b658-9be38f90d8b1 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:11.672871 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d771ed-eef8-4b93-96e2-85c4b8ab74b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:11.673531 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d771ed-eef8-4b93-96e2-85c4b8ab74b6 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:11.673816 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d771ed-eef8-4b93-96e2-85c4b8ab74b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:11.674110 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d771ed-eef8-4b93-96e2-85c4b8ab74b6 None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:11.674843 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d771ed-eef8-4b93-96e2-85c4b8ab74b6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:11.675126 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cd80a0-6bca-4088-ad06-a37b4799723a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:11.675252 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d771ed-eef8-4b93-96e2-85c4b8ab74b6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:11.676057 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cd80a0-6bca-4088-ad06-a37b4799723a None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:11.676527 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cd80a0-6bca-4088-ad06-a37b4799723a None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:11.676694 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d771ed-eef8-4b93-96e2-85c4b8ab74b6 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:11.677168 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cd80a0-6bca-4088-ad06-a37b4799723a None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:11.678053 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cd80a0-6bca-4088-ad06-a37b4799723a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:11.678821 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cd80a0-6bca-4088-ad06-a37b4799723a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:11.680999 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cd80a0-6bca-4088-ad06-a37b4799723a None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:11.700747 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 413/837] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Aug 6 01:21:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:11.700747 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d771ed-eef8-4b93-96e2-85c4b8ab74b6 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:11.701283 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 424/837] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Aug 6 01:21:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:11.702640 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cd80a0-6bca-4088-ad06-a37b4799723a None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:11.759484 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:11.772760 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:11.813086 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cd80a0-6bca-4088-ad06-a37b4799723a None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:11.838307 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d771ed-eef8-4b93-96e2-85c4b8ab74b6 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:11.849619 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96f2f83-c627-4c0f-94dd-1e22062d81ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:11.850401 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96f2f83-c627-4c0f-94dd-1e22062d81ed None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:11.850777 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96f2f83-c627-4c0f-94dd-1e22062d81ed None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:11.851444 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96f2f83-c627-4c0f-94dd-1e22062d81ed None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:11.858857 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96f2f83-c627-4c0f-94dd-1e22062d81ed None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:11.860389 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96f2f83-c627-4c0f-94dd-1e22062d81ed None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:11.872858 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-d96f2f83-c627-4c0f-94dd-1e22062d81ed None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 06 01:21:11.872858 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:11.872858 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:21:11.872858 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:21:11.872858 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:21:11.872858 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:21:11.872858 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:21:11.872858 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:21:11.872858 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:21:11.872858 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:21:11.872858 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:21:11.872858 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:21:11.872858 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 01:21:11.872858 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 01:21:11.872858 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 01:21:11.872858 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 01:21:11.874036 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:21:11.874036 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:21:11.874036 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 01:21:11.874036 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:21:11.874036 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 01:21:11.874036 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:21:11.874036 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:21:11.874036 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:21:11.874036 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:21:11.874036 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:21:11.874036 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:21:11.874036 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:21:11.874036 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:21:11.874036 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:21:11.874036 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:21:11.874036 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:21:11.874036 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:21:11.874036 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:21:11.875204 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:21:11.875204 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:21:11.875204 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 01:21:11.875204 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 01:21:11.875204 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 01:21:11.875204 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 01:21:11.875204 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 01:21:11.875204 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 01:21:11.875204 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 06 01:21:11.875204 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:21:11.876253 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 425/838] 10.222.0.26 () {56 vars in 1117 bytes} [Sat Aug 6 01:21:11 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:21:11.877561 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e2cece-5a78-4e41-9097-36d15376eb6d None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:11.877936 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e2cece-5a78-4e41-9097-36d15376eb6d None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:11.878220 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e2cece-5a78-4e41-9097-36d15376eb6d None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:11.878801 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9586abc-4485-43e6-93ee-fa8b272094ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:11.879562 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9586abc-4485-43e6-93ee-fa8b272094ad None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:11.879710 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 426/839] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:21:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:21:11.880107 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9586abc-4485-43e6-93ee-fa8b272094ad None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:11.880390 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9586abc-4485-43e6-93ee-fa8b272094ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:11.882270 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9586abc-4485-43e6-93ee-fa8b272094ad None admin] RBAC: Adding query filter params (role.id=4559b9a0837249f895a13c70096f8d38, scope.project.id=d9d68d310bf24449a5fe20d2dc264a58, user.id=b87adc1d5bc240e99f95f6595b350be4) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:11.882270 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9586abc-4485-43e6-93ee-fa8b272094ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:11.882956 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9586abc-4485-43e6-93ee-fa8b272094ad None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:11.884193 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e2cece-5a78-4e41-9097-36d15376eb6d None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:11.890963 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 414/840] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Aug 6 01:21:11 2022] GET /identity/v3/role_assignments?user.id=b87adc1d5bc240e99f95f6595b350be4&scope.project.id=d9d68d310bf24449a5fe20d2dc264a58&role.id=4559b9a0837249f895a13c70096f8d38 => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:21:11.892374 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349131d8-c07c-4307-8dce-89a62f79c888 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:11.892811 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349131d8-c07c-4307-8dce-89a62f79c888 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:11.893266 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349131d8-c07c-4307-8dce-89a62f79c888 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:11.913102 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-349131d8-c07c-4307-8dce-89a62f79c888 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:21:11.914573 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-349131d8-c07c-4307-8dce-89a62f79c888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:21:11.997477 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 415/841] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:21:11 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:21:12.011588 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e2cece-5a78-4e41-9097-36d15376eb6d None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:12.014896 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349131d8-c07c-4307-8dce-89a62f79c888 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:12.027791 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:12.055465 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cc0889-601c-4475-866f-f0e883d92da4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:12.056867 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cc0889-601c-4475-866f-f0e883d92da4 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:12.056867 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cc0889-601c-4475-866f-f0e883d92da4 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:12.056867 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cc0889-601c-4475-866f-f0e883d92da4 None admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:12.058041 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cc0889-601c-4475-866f-f0e883d92da4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:12.058339 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cc0889-601c-4475-866f-f0e883d92da4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:12.059876 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cc0889-601c-4475-866f-f0e883d92da4 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:12.074498 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:12.080469 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 427/842] 10.222.0.26 () {56 vars in 1149 bytes} [Sat Aug 6 01:21:11 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:21:12.081812 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc661a0-37ae-457c-a2ee-b96203ff7c77 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:12.082105 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc661a0-37ae-457c-a2ee-b96203ff7c77 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:12.082386 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc661a0-37ae-457c-a2ee-b96203ff7c77 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:12.083594 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 428/843] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:21:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:21:12.086429 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc661a0-37ae-457c-a2ee-b96203ff7c77 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:12.145117 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349131d8-c07c-4307-8dce-89a62f79c888 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:12.181844 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c13e58-51f4-4e25-8587-c44ebf95914d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:12.183237 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c13e58-51f4-4e25-8587-c44ebf95914d None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:12.183645 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c13e58-51f4-4e25-8587-c44ebf95914d None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:12.183927 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c13e58-51f4-4e25-8587-c44ebf95914d None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:12.184690 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c13e58-51f4-4e25-8587-c44ebf95914d None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:12.186321 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c13e58-51f4-4e25-8587-c44ebf95914d None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:12.196187 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-d3c13e58-51f4-4e25-8587-c44ebf95914d None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 06 01:21:12.196187 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:12.196187 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 01:21:12.196187 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 01:21:12.196187 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 01:21:12.196187 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 01:21:12.196187 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 01:21:12.196187 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 01:21:12.196187 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 06 01:21:12.196187 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:21:12.196187 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 01:21:12.196187 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:21:12.196187 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:12.196187 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:21:12.196187 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:21:12.196187 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:21:12.196187 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:21:12.197205 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:21:12.197205 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:21:12.197205 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:21:12.197205 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:21:12.197205 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:21:12.197205 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:21:12.197205 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 01:21:12.197205 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 01:21:12.197205 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 01:21:12.197205 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 01:21:12.197205 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:21:12.197205 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:21:12.197205 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:21:12.197205 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:21:12.197205 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:21:12.197205 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:21:12.198088 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:21:12.198088 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:21:12.198088 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:21:12.198088 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:21:12.198088 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:21:12.198088 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:21:12.198088 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:21:12.198088 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:21:12.198088 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:21:12.198088 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:21:12.198088 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 01:21:12.198088 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 01:21:12.198088 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 06 01:21:12.198088 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:21:12.198088 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 416/844] 10.222.0.26 () {56 vars in 1114 bytes} [Sat Aug 6 01:21:12 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:21:12.198972 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fc5fdb-44c6-4aa5-bea0-1c9f1310a98b None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:12.199052 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fc5fdb-44c6-4aa5-bea0-1c9f1310a98b None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:12.199637 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fc5fdb-44c6-4aa5-bea0-1c9f1310a98b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:12.206068 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc661a0-37ae-457c-a2ee-b96203ff7c77 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:12.219669 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-61fc5fdb-44c6-4aa5-bea0-1c9f1310a98b None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:21:12.220860 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-61fc5fdb-44c6-4aa5-bea0-1c9f1310a98b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:21:12.244326 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8f9396-cd3e-4a73-9059-04b5b65e73ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:12.245118 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8f9396-cd3e-4a73-9059-04b5b65e73ee None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:12.245515 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8f9396-cd3e-4a73-9059-04b5b65e73ee None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:12.245798 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8f9396-cd3e-4a73-9059-04b5b65e73ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:12.289271 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:12.294034 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 417/845] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:21:12 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:21:12.294905 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fc5fdb-44c6-4aa5-bea0-1c9f1310a98b None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:12.328886 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8f9396-cd3e-4a73-9059-04b5b65e73ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:12.330305 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8f9396-cd3e-4a73-9059-04b5b65e73ee None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:12.343830 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:12.421730 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fc5fdb-44c6-4aa5-bea0-1c9f1310a98b None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:12.441227 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 429/846] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Aug 6 01:21:12 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 06 01:21:12.442407 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8f9396-cd3e-4a73-9059-04b5b65e73ee None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:12.461235 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7981bf8-4e40-405d-972a-41a0d6e5a1d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:12.461877 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7981bf8-4e40-405d-972a-41a0d6e5a1d6 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:12.462150 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7981bf8-4e40-405d-972a-41a0d6e5a1d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:12.462426 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7981bf8-4e40-405d-972a-41a0d6e5a1d6 None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:12.463104 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7981bf8-4e40-405d-972a-41a0d6e5a1d6 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:12.463987 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7981bf8-4e40-405d-972a-41a0d6e5a1d6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:12.465768 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7981bf8-4e40-405d-972a-41a0d6e5a1d6 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:12.470724 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 418/847] 10.222.0.26 () {56 vars in 1114 bytes} [Sat Aug 6 01:21:12 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:12.471502 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7981bf8-4e40-405d-972a-41a0d6e5a1d6 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:12.522905 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:12.559060 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8f9396-cd3e-4a73-9059-04b5b65e73ee None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:12.579729 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7981bf8-4e40-405d-972a-41a0d6e5a1d6 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:12.581422 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:12.593962 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f810dc59-701b-4a9c-93ca-3344a18a4aee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:12.594607 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f810dc59-701b-4a9c-93ca-3344a18a4aee None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:12.594890 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f810dc59-701b-4a9c-93ca-3344a18a4aee None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:12.595162 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f810dc59-701b-4a9c-93ca-3344a18a4aee None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:12.606228 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f810dc59-701b-4a9c-93ca-3344a18a4aee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:21:12.611252 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-f810dc59-701b-4a9c-93ca-3344a18a4aee None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:21:12.611252 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:12.611252 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:21:12.611252 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:21:12.611252 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:21:12.611252 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:21:12.611252 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:21:12.611252 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:21:12.611252 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:21:12.611252 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:21:12.611252 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:21:12.611252 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:21:12.611252 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:21:12.611252 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:21:12.611252 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:21:12.611252 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:21:12.612193 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:21:12.612193 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:21:12.612193 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:21:12.612193 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:21:12.612193 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:21:12.612193 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:21:12.612193 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:21:12.612193 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:21:12.612193 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:21:12.612193 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:21:12.612193 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:21:12.612193 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:21:12.612193 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:21:12.612193 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:21:12.612193 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:21:12.612193 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:21:12.613142 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:21:12.613142 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:21:12.613142 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:21:12.613142 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:21:12.613142 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:21:12.613142 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:21:12.613142 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:21:12.613142 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:21:12.613142 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:21:12.613142 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:21:12.613142 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:21:12.613142 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:21:12.613142 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 430/848] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Aug 6 01:21:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:21:12.613873 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f810dc59-701b-4a9c-93ca-3344a18a4aee None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:12.627994 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e51f67-6428-43b7-a722-632ad202e17c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:12.628641 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e51f67-6428-43b7-a722-632ad202e17c None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:12.628925 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e51f67-6428-43b7-a722-632ad202e17c None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:12.629237 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e51f67-6428-43b7-a722-632ad202e17c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:12.640285 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e51f67-6428-43b7-a722-632ad202e17c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:12.641755 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e51f67-6428-43b7-a722-632ad202e17c None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:12.656345 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-c6e51f67-6428-43b7-a722-632ad202e17c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:21:12.656345 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:12.656345 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:21:12.656345 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:21:12.656345 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:21:12.656345 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:21:12.656345 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:21:12.656345 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:21:12.656345 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:21:12.656345 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:21:12.656345 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:21:12.656345 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:21:12.656345 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:21:12.656345 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:21:12.656345 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:21:12.657300 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:21:12.657300 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:21:12.657300 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:21:12.657300 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:21:12.657300 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:21:12.657300 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:21:12.657300 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:21:12.657300 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:21:12.657300 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:21:12.657300 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:21:12.657300 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:21:12.657300 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:21:12.657300 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:21:12.657300 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:21:12.657300 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:21:12.657300 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:21:12.657300 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:21:12.658224 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:21:12.658224 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:21:12.658224 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:21:12.658224 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:21:12.662317 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 419/849] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:21:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:21:12.663185 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e51f67-6428-43b7-a722-632ad202e17c None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:12.727540 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f810dc59-701b-4a9c-93ca-3344a18a4aee None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:12.759581 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:12.767557 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f5bc07-2a8f-4b53-813b-3f6421b31b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:12.768149 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f5bc07-2a8f-4b53-813b-3f6421b31b22 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:12.768432 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f5bc07-2a8f-4b53-813b-3f6421b31b22 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:12.768704 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f5bc07-2a8f-4b53-813b-3f6421b31b22 None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:12.769412 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f5bc07-2a8f-4b53-813b-3f6421b31b22 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:12.769718 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f5bc07-2a8f-4b53-813b-3f6421b31b22 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:12.771085 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f5bc07-2a8f-4b53-813b-3f6421b31b22 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:12.772830 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e51f67-6428-43b7-a722-632ad202e17c None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:12.775737 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 431/850] 10.222.0.26 () {56 vars in 1114 bytes} [Sat Aug 6 01:21:12 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:12.776429 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f5bc07-2a8f-4b53-813b-3f6421b31b22 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:12.829452 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94adee8a-607e-4f8f-b5bb-a9dc5c39c707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:12.830035 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94adee8a-607e-4f8f-b5bb-a9dc5c39c707 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:12.830320 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94adee8a-607e-4f8f-b5bb-a9dc5c39c707 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:12.830596 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94adee8a-607e-4f8f-b5bb-a9dc5c39c707 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:12.831554 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94adee8a-607e-4f8f-b5bb-a9dc5c39c707 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:12.831688 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94adee8a-607e-4f8f-b5bb-a9dc5c39c707 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:12.833084 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94adee8a-607e-4f8f-b5bb-a9dc5c39c707 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:12.843624 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 420/851] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Aug 6 01:21:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:12.848844 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94adee8a-607e-4f8f-b5bb-a9dc5c39c707 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:12.871222 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:12.915714 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f5bc07-2a8f-4b53-813b-3f6421b31b22 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:12.966193 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5adee14-e938-46ec-9abd-9eef4d98e427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:12.966827 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5adee14-e938-46ec-9abd-9eef4d98e427 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:12.967114 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5adee14-e938-46ec-9abd-9eef4d98e427 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:12.967404 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5adee14-e938-46ec-9abd-9eef4d98e427 None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:12.968101 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5adee14-e938-46ec-9abd-9eef4d98e427 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:12.968409 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5adee14-e938-46ec-9abd-9eef4d98e427 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:12.969855 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5adee14-e938-46ec-9abd-9eef4d98e427 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:12.982521 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94adee8a-607e-4f8f-b5bb-a9dc5c39c707 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:12.987300 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 432/852] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Aug 6 01:21:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:12.993005 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5adee14-e938-46ec-9abd-9eef4d98e427 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:13.022259 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b873d18f-2ad2-4fa8-ae41-c793ba775714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:13.022906 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b873d18f-2ad2-4fa8-ae41-c793ba775714 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:13.023334 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b873d18f-2ad2-4fa8-ae41-c793ba775714 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:13.023578 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b873d18f-2ad2-4fa8-ae41-c793ba775714 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:13.035871 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b873d18f-2ad2-4fa8-ae41-c793ba775714 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:13.037476 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b873d18f-2ad2-4fa8-ae41-c793ba775714 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:13.053860 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-b873d18f-2ad2-4fa8-ae41-c793ba775714 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:21:13.053860 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:13.053860 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:21:13.053860 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:21:13.053860 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:21:13.053860 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:21:13.053860 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:21:13.053860 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:21:13.053860 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:21:13.053860 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:21:13.053860 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:21:13.053860 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:21:13.053860 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:21:13.053860 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:21:13.053860 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:21:13.055004 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:21:13.055004 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:21:13.055004 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:21:13.055004 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:21:13.055004 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:21:13.055004 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:21:13.055004 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:21:13.055004 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:21:13.055004 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:21:13.055004 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:21:13.055004 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:21:13.055004 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:21:13.055004 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:21:13.055004 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:21:13.055004 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:21:13.055004 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:21:13.055004 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:21:13.056051 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:21:13.056051 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:21:13.056051 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:21:13.056051 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:21:13.059962 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 421/853] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:21:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:21:13.061028 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b873d18f-2ad2-4fa8-ae41-c793ba775714 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:13.062844 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:13.121513 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5adee14-e938-46ec-9abd-9eef4d98e427 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:13.154264 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:13.173692 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cecfb5-1111-4ea2-ba20-03b3f273cc1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:13.174783 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cecfb5-1111-4ea2-ba20-03b3f273cc1c None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:13.175169 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cecfb5-1111-4ea2-ba20-03b3f273cc1c None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:13.175548 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cecfb5-1111-4ea2-ba20-03b3f273cc1c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:13.185358 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cecfb5-1111-4ea2-ba20-03b3f273cc1c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:13.187583 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cecfb5-1111-4ea2-ba20-03b3f273cc1c None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:13.193860 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b873d18f-2ad2-4fa8-ae41-c793ba775714 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:13.202769 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-e2cecfb5-1111-4ea2-ba20-03b3f273cc1c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:21:13.202769 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:13.202769 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:21:13.202769 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:21:13.202769 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:21:13.202769 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:21:13.202769 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:21:13.202769 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:21:13.202769 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:21:13.202769 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:21:13.202769 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:21:13.202769 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:21:13.202769 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 01:21:13.202769 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 01:21:13.202769 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 01:21:13.203860 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 01:21:13.203860 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:21:13.203860 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:21:13.203860 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:21:13.203860 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:21:13.203860 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:21:13.203860 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:21:13.203860 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:21:13.203860 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:21:13.203860 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:21:13.203860 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:21:13.203860 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:21:13.203860 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:21:13.203860 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:21:13.203860 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:21:13.203860 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:21:13.203860 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:21:13.204936 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 01:21:13.204936 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 01:21:13.204936 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 01:21:13.204936 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 01:21:13.204936 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 01:21:13.204936 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 01:21:13.204936 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:21:13.204936 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:21:13.209138 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 433/854] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Aug 6 01:21:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:21:13.212353 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cecfb5-1111-4ea2-ba20-03b3f273cc1c None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:13.240264 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128740cd-e19d-4787-965c-3053904935cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:13.241092 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128740cd-e19d-4787-965c-3053904935cb None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:13.241498 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128740cd-e19d-4787-965c-3053904935cb None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:13.241877 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128740cd-e19d-4787-965c-3053904935cb None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:13.242729 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128740cd-e19d-4787-965c-3053904935cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:13.243509 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128740cd-e19d-4787-965c-3053904935cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:13.244603 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128740cd-e19d-4787-965c-3053904935cb None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:13.258157 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 422/855] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Aug 6 01:21:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:21:13.263727 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128740cd-e19d-4787-965c-3053904935cb None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:13.336696 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cecfb5-1111-4ea2-ba20-03b3f273cc1c None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:13.353812 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:13.377309 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5f1ca9-9d02-4015-a2fd-451557902b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:13.378197 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5f1ca9-9d02-4015-a2fd-451557902b57 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:13.378364 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5f1ca9-9d02-4015-a2fd-451557902b57 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:13.378738 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5f1ca9-9d02-4015-a2fd-451557902b57 None admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:13.379646 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5f1ca9-9d02-4015-a2fd-451557902b57 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:13.380055 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5f1ca9-9d02-4015-a2fd-451557902b57 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:13.381613 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5f1ca9-9d02-4015-a2fd-451557902b57 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:13.396388 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128740cd-e19d-4787-965c-3053904935cb None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:13.399820 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 434/856] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Aug 6 01:21:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:21:13.400643 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5f1ca9-9d02-4015-a2fd-451557902b57 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:13.439064 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:13.450495 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a57ff27-77c3-4906-9a05-fb0bd9bb185c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:13.451373 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a57ff27-77c3-4906-9a05-fb0bd9bb185c None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:13.452249 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a57ff27-77c3-4906-9a05-fb0bd9bb185c None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:13.452706 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a57ff27-77c3-4906-9a05-fb0bd9bb185c None admin] PATH_INFO: `/v3/users/swift` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:13.464324 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a57ff27-77c3-4906-9a05-fb0bd9bb185c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:13.466020 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a57ff27-77c3-4906-9a05-fb0bd9bb185c None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:13.484844 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-6a57ff27-77c3-4906-9a05-fb0bd9bb185c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 06 01:21:13.484844 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:13.484844 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:21:13.484844 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:21:13.484844 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:21:13.484844 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:21:13.484844 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:21:13.484844 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:21:13.484844 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:21:13.484844 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:21:13.484844 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:21:13.484844 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:21:13.484844 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 01:21:13.484844 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 01:21:13.484844 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 01:21:13.484844 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 01:21:13.486043 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:21:13.486043 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:21:13.486043 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 01:21:13.486043 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:21:13.486043 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 01:21:13.486043 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:21:13.486043 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:21:13.486043 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:21:13.486043 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:21:13.486043 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:21:13.486043 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:21:13.486043 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:21:13.486043 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:21:13.486043 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:21:13.486043 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:21:13.486043 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:21:13.486043 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:21:13.486043 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:21:13.487181 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:21:13.487181 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:21:13.487181 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 01:21:13.487181 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 01:21:13.487181 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 01:21:13.487181 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 01:21:13.487181 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 01:21:13.487181 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 01:21:13.487181 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 06 01:21:13.487181 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:21:13.490485 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 423/857] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Aug 6 01:21:13 2022] GET /identity/v3/users/swift => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:21:13.496241 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a57ff27-77c3-4906-9a05-fb0bd9bb185c None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:13.532915 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5f1ca9-9d02-4015-a2fd-451557902b57 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:13.580874 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13087776-c068-474c-ba68-314d0e27de0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:13.581732 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13087776-c068-474c-ba68-314d0e27de0a None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:13.582165 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13087776-c068-474c-ba68-314d0e27de0a None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:13.582781 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13087776-c068-474c-ba68-314d0e27de0a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:13.583473 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13087776-c068-474c-ba68-314d0e27de0a None admin] RBAC: Adding query filter params (role.id=c9ce798acd4b482f8f391c6af97eb7a4, scope.project.id=d9d68d310bf24449a5fe20d2dc264a58, user.id=8e86f38590824894a3b960e6774ff8a9) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:13.583884 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13087776-c068-474c-ba68-314d0e27de0a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:13.585464 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13087776-c068-474c-ba68-314d0e27de0a None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:13.597346 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 435/858] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Aug 6 01:21:13 2022] GET /identity/v3/role_assignments?user.id=8e86f38590824894a3b960e6774ff8a9&scope.project.id=d9d68d310bf24449a5fe20d2dc264a58&role.id=c9ce798acd4b482f8f391c6af97eb7a4 => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:13.613344 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a57ff27-77c3-4906-9a05-fb0bd9bb185c None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:13.653744 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a882842c-d105-4b75-a48f-8bec9fa2a17e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:13.655058 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a882842c-d105-4b75-a48f-8bec9fa2a17e None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:13.655493 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a882842c-d105-4b75-a48f-8bec9fa2a17e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:13.655941 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a882842c-d105-4b75-a48f-8bec9fa2a17e None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:13.656906 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a882842c-d105-4b75-a48f-8bec9fa2a17e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:13.657382 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a882842c-d105-4b75-a48f-8bec9fa2a17e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:13.659004 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a882842c-d105-4b75-a48f-8bec9fa2a17e None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:13.680429 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 424/859] 10.222.0.26 () {56 vars in 1135 bytes} [Sat Aug 6 01:21:13 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:13.687051 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13087776-c068-474c-ba68-314d0e27de0a None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:13.733999 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:13.797030 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13087776-c068-474c-ba68-314d0e27de0a None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:13.831845 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306c52aa-5b38-462a-b548-df3b420be9f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:13.832713 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306c52aa-5b38-462a-b548-df3b420be9f4 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:13.833234 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306c52aa-5b38-462a-b548-df3b420be9f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:13.833660 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306c52aa-5b38-462a-b548-df3b420be9f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:13.905653 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306c52aa-5b38-462a-b548-df3b420be9f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:13.907320 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306c52aa-5b38-462a-b548-df3b420be9f4 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:13.947325 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:14.013945 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 436/860] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Aug 6 01:21:13 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 06 01:21:14.019691 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a882842c-d105-4b75-a48f-8bec9fa2a17e None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:14.042252 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894b4c5e-1f78-4c6a-aa51-3f9f2da9a1a7 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:14.042449 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894b4c5e-1f78-4c6a-aa51-3f9f2da9a1a7 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:14.042723 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894b4c5e-1f78-4c6a-aa51-3f9f2da9a1a7 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:14.044786 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 437/861] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:21:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:21:14.050247 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d472197-d205-4493-8b44-474b3d7c0b77 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:14.050573 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d472197-d205-4493-8b44-474b3d7c0b77 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:14.050909 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d472197-d205-4493-8b44-474b3d7c0b77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:14.069736 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-3d472197-d205-4493-8b44-474b3d7c0b77 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:21:14.071795 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-3d472197-d205-4493-8b44-474b3d7c0b77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:21:14.078307 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:14.139247 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a882842c-d105-4b75-a48f-8bec9fa2a17e None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:14.155454 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 438/862] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:21:14 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 06 01:21:14.176144 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3865a7e7-518b-42e7-be95-43be632b3e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:14.177111 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d472197-d205-4493-8b44-474b3d7c0b77 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:14.177986 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3865a7e7-518b-42e7-be95-43be632b3e79 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:14.178582 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3865a7e7-518b-42e7-be95-43be632b3e79 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:14.179068 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3865a7e7-518b-42e7-be95-43be632b3e79 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:14.190057 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3865a7e7-518b-42e7-be95-43be632b3e79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:14.191836 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3865a7e7-518b-42e7-be95-43be632b3e79 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:14.207691 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-3865a7e7-518b-42e7-be95-43be632b3e79 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:21:14.207691 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:14.207691 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:21:14.207691 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:21:14.207691 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:21:14.207691 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:21:14.207691 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:21:14.207691 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:21:14.207691 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:21:14.207691 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:21:14.207691 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:21:14.207691 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:21:14.207691 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:21:14.207691 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:21:14.207691 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:21:14.208625 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:21:14.208625 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:21:14.208625 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:21:14.208625 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:21:14.208625 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:21:14.208625 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:21:14.208625 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:21:14.208625 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:21:14.208625 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:21:14.208625 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:21:14.208625 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:21:14.208625 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:21:14.208625 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:21:14.208625 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:21:14.208625 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:21:14.208625 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:21:14.208625 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:21:14.209568 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:21:14.209568 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:21:14.209568 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:21:14.209568 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:21:14.211040 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 425/863] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:21:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:21:14.216778 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3865a7e7-518b-42e7-be95-43be632b3e79 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:14.256847 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:14.299587 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d472197-d205-4493-8b44-474b3d7c0b77 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:14.344611 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fc177e-53d5-46e0-a8ad-e754523985f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:14.345337 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fc177e-53d5-46e0-a8ad-e754523985f9 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:14.345554 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fc177e-53d5-46e0-a8ad-e754523985f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:14.345826 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fc177e-53d5-46e0-a8ad-e754523985f9 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:14.346455 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fc177e-53d5-46e0-a8ad-e754523985f9 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:14.347865 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fc177e-53d5-46e0-a8ad-e754523985f9 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:14.353376 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:14.358202 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-88fc177e-53d5-46e0-a8ad-e754523985f9 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 06 01:21:14.358202 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:14.358202 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 01:21:14.358202 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 01:21:14.358202 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 01:21:14.358202 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 01:21:14.358202 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 01:21:14.358202 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 01:21:14.358202 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 06 01:21:14.358202 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:21:14.358202 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 01:21:14.358202 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:21:14.358202 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:14.358202 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:21:14.358202 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:21:14.358202 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:21:14.359387 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:21:14.359387 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:21:14.359387 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:21:14.359387 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:21:14.359387 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:21:14.359387 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:21:14.359387 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:21:14.359387 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 01:21:14.359387 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 01:21:14.359387 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 01:21:14.359387 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 01:21:14.359387 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:21:14.359387 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:21:14.359387 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:21:14.359387 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:21:14.359387 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:21:14.360428 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:21:14.360428 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:21:14.360428 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:21:14.360428 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:21:14.360428 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:21:14.360428 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:21:14.360428 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:21:14.360428 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:21:14.360428 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:21:14.360428 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:21:14.360428 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:21:14.360428 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 01:21:14.360428 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 01:21:14.360428 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 06 01:21:14.360428 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:21:14.363253 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3865a7e7-518b-42e7-be95-43be632b3e79 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:14.364328 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 439/864] 10.222.0.26 () {56 vars in 1132 bytes} [Sat Aug 6 01:21:14 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:21:14.365189 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fc177e-53d5-46e0-a8ad-e754523985f9 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:14.413359 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8add54b3-ab8f-4256-b0cf-666bbb9cebdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:14.414241 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8add54b3-ab8f-4256-b0cf-666bbb9cebdc None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:14.414677 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8add54b3-ab8f-4256-b0cf-666bbb9cebdc None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:14.415103 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8add54b3-ab8f-4256-b0cf-666bbb9cebdc None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:14.415994 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8add54b3-ab8f-4256-b0cf-666bbb9cebdc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:14.417151 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8add54b3-ab8f-4256-b0cf-666bbb9cebdc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:14.418693 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8add54b3-ab8f-4256-b0cf-666bbb9cebdc None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:14.440936 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 426/865] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Aug 6 01:21:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:14.441755 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8add54b3-ab8f-4256-b0cf-666bbb9cebdc None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:14.502631 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fc177e-53d5-46e0-a8ad-e754523985f9 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:14.536459 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:14.543009 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b37a6ce-ee42-41a6-81fe-6f8efe29985c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:14.544027 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b37a6ce-ee42-41a6-81fe-6f8efe29985c None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:14.544404 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b37a6ce-ee42-41a6-81fe-6f8efe29985c None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:14.544776 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b37a6ce-ee42-41a6-81fe-6f8efe29985c None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:14.545628 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b37a6ce-ee42-41a6-81fe-6f8efe29985c None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:14.546000 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b37a6ce-ee42-41a6-81fe-6f8efe29985c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:14.547521 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b37a6ce-ee42-41a6-81fe-6f8efe29985c None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:14.558353 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 440/866] 10.222.0.26 () {56 vars in 1126 bytes} [Sat Aug 6 01:21:14 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:14.559547 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b37a6ce-ee42-41a6-81fe-6f8efe29985c None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:14.565786 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8add54b3-ab8f-4256-b0cf-666bbb9cebdc None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:14.616198 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde800a9-b8a9-46b4-888d-2ef97dec29c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:14.617106 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde800a9-b8a9-46b4-888d-2ef97dec29c2 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:14.617554 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde800a9-b8a9-46b4-888d-2ef97dec29c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:14.618542 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde800a9-b8a9-46b4-888d-2ef97dec29c2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:14.629443 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde800a9-b8a9-46b4-888d-2ef97dec29c2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:14.631175 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde800a9-b8a9-46b4-888d-2ef97dec29c2 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:14.650177 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-fde800a9-b8a9-46b4-888d-2ef97dec29c2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:21:14.650177 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:14.650177 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:21:14.650177 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:21:14.650177 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:21:14.650177 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:21:14.650177 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:21:14.650177 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:21:14.650177 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:21:14.650177 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:21:14.650177 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:21:14.650177 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:21:14.650177 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 01:21:14.650177 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 01:21:14.650177 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 01:21:14.652037 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 01:21:14.652037 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:21:14.652037 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:21:14.652037 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:21:14.652037 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:21:14.652037 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:21:14.652037 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:21:14.652037 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:21:14.652037 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:21:14.652037 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:21:14.652037 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:21:14.652037 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:21:14.652037 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:21:14.652037 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:21:14.652037 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:21:14.652037 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:21:14.652037 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:21:14.653188 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 01:21:14.653188 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 01:21:14.653188 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 01:21:14.653188 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 01:21:14.653188 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 01:21:14.653188 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 01:21:14.653188 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:21:14.653188 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:21:14.659690 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 427/867] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Aug 6 01:21:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:21:14.661873 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde800a9-b8a9-46b4-888d-2ef97dec29c2 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:14.679023 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b37a6ce-ee42-41a6-81fe-6f8efe29985c None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:14.688108 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:14.723764 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f466088-1cf4-4abd-93be-37b969fe2740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:14.725028 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f466088-1cf4-4abd-93be-37b969fe2740 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:14.725755 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f466088-1cf4-4abd-93be-37b969fe2740 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:14.726447 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f466088-1cf4-4abd-93be-37b969fe2740 None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:14.727439 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f466088-1cf4-4abd-93be-37b969fe2740 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:14.727921 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f466088-1cf4-4abd-93be-37b969fe2740 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:14.729869 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f466088-1cf4-4abd-93be-37b969fe2740 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:14.736530 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 441/868] 10.222.0.26 () {56 vars in 1126 bytes} [Sat Aug 6 01:21:14 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:14.795509 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde800a9-b8a9-46b4-888d-2ef97dec29c2 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:14.841331 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccc3116-9b66-4aca-b7c7-dae436a8cf1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:14.841969 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccc3116-9b66-4aca-b7c7-dae436a8cf1f None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:14.842249 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccc3116-9b66-4aca-b7c7-dae436a8cf1f None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:14.842524 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccc3116-9b66-4aca-b7c7-dae436a8cf1f None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:14.843283 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccc3116-9b66-4aca-b7c7-dae436a8cf1f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:14.843593 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccc3116-9b66-4aca-b7c7-dae436a8cf1f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:14.845028 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccc3116-9b66-4aca-b7c7-dae436a8cf1f None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:14.859915 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 428/869] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Aug 6 01:21:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:14.865950 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f466088-1cf4-4abd-93be-37b969fe2740 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:14.869151 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f746e1-1260-49e8-a06b-0762013e3f8b None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:14.869446 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f746e1-1260-49e8-a06b-0762013e3f8b None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:14.869773 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f746e1-1260-49e8-a06b-0762013e3f8b None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:14.875466 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 429/870] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:21:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:21:14.877321 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713095c0-c00b-4949-a88f-3d3282129dda None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:14.877615 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713095c0-c00b-4949-a88f-3d3282129dda None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:14.877899 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713095c0-c00b-4949-a88f-3d3282129dda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:14.898060 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-713095c0-c00b-4949-a88f-3d3282129dda None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:21:14.899334 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-713095c0-c00b-4949-a88f-3d3282129dda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:21:14.946732 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:14.989253 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 430/871] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:21:14 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 06 01:21:14.998033 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f466088-1cf4-4abd-93be-37b969fe2740 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:15.007801 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713095c0-c00b-4949-a88f-3d3282129dda None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:15.019116 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:15.038690 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713f547f-3787-4ac0-8e2a-b514f8a831c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:15.039864 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713f547f-3787-4ac0-8e2a-b514f8a831c6 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:15.040468 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713f547f-3787-4ac0-8e2a-b514f8a831c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:15.041061 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713f547f-3787-4ac0-8e2a-b514f8a831c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:15.042710 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713f547f-3787-4ac0-8e2a-b514f8a831c6 None admin] RBAC: Adding query filter params (role.id=4559b9a0837249f895a13c70096f8d38, scope.project.id=d9d68d310bf24449a5fe20d2dc264a58, user.id=2b23f4478f49471fba6f43e6e4be99da) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:15.043205 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713f547f-3787-4ac0-8e2a-b514f8a831c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:15.044990 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713f547f-3787-4ac0-8e2a-b514f8a831c6 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:15.051710 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 442/872] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Aug 6 01:21:14 2022] GET /identity/v3/role_assignments?user.id=2b23f4478f49471fba6f43e6e4be99da&scope.project.id=d9d68d310bf24449a5fe20d2dc264a58&role.id=4559b9a0837249f895a13c70096f8d38 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:15.127134 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713095c0-c00b-4949-a88f-3d3282129dda None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:15.165719 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16329852-0f4c-49d8-8f10-a8204098d96c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:15.167200 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16329852-0f4c-49d8-8f10-a8204098d96c None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:15.167630 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16329852-0f4c-49d8-8f10-a8204098d96c None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:15.168189 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16329852-0f4c-49d8-8f10-a8204098d96c None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:15.169117 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16329852-0f4c-49d8-8f10-a8204098d96c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:15.170675 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16329852-0f4c-49d8-8f10-a8204098d96c None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:15.193600 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 431/873] 10.222.0.26 () {60 vars in 1141 bytes} [Sat Aug 6 01:21:15 2022] POST /identity/v3/services => generated 251 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:21:15.837951 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5e02ad-00f1-4046-ab80-918b29e2c58d None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:15.839314 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5e02ad-00f1-4046-ab80-918b29e2c58d None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:15.839314 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5e02ad-00f1-4046-ab80-918b29e2c58d None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:15.840840 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 443/874] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:21:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:21:15.846356 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419882aa-8e6c-4f03-a3d7-7ba6a10b5edc None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:15.846770 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419882aa-8e6c-4f03-a3d7-7ba6a10b5edc None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:15.846959 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419882aa-8e6c-4f03-a3d7-7ba6a10b5edc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:15.865905 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-419882aa-8e6c-4f03-a3d7-7ba6a10b5edc None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:21:15.867389 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-419882aa-8e6c-4f03-a3d7-7ba6a10b5edc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:21:15.934744 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:15.948761 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 432/875] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:21:15 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:21:15.972310 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5e02ad-00f1-4046-ab80-918b29e2c58d None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:16.083418 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5e02ad-00f1-4046-ab80-918b29e2c58d None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:16.108825 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:16.120284 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722d519a-54f0-40b0-ac22-1bbe5aa9ab9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:16.120928 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722d519a-54f0-40b0-ac22-1bbe5aa9ab9b None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:16.121250 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722d519a-54f0-40b0-ac22-1bbe5aa9ab9b None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:16.121539 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722d519a-54f0-40b0-ac22-1bbe5aa9ab9b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:16.133707 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722d519a-54f0-40b0-ac22-1bbe5aa9ab9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:21:16.139431 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-722d519a-54f0-40b0-ac22-1bbe5aa9ab9b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 06 01:21:16.139431 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:16.139431 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:21:16.139431 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:21:16.139431 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:21:16.139431 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:21:16.139431 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:21:16.139431 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:21:16.139431 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:21:16.139431 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:21:16.139431 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:21:16.139431 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:21:16.139431 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:21:16.139431 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:21:16.139431 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:21:16.139431 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:21:16.140390 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:21:16.140390 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:21:16.140390 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:21:16.140390 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:21:16.140390 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:21:16.140390 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:21:16.140390 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:21:16.140390 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:21:16.140390 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:21:16.140390 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:21:16.140390 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:21:16.140390 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:21:16.140390 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:21:16.140390 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:21:16.140390 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:21:16.140390 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:21:16.141705 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:21:16.141705 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:21:16.141705 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:21:16.141705 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:21:16.141705 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:21:16.141705 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:21:16.141705 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:21:16.141705 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:21:16.141705 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:21:16.141705 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:21:16.141705 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 06 01:21:16.141705 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:21:16.141705 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 444/876] 10.222.0.26 () {56 vars in 1120 bytes} [Sat Aug 6 01:21:15 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:21:16.150152 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419882aa-8e6c-4f03-a3d7-7ba6a10b5edc None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:16.258686 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419882aa-8e6c-4f03-a3d7-7ba6a10b5edc None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:16.295178 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657aeb9a-ca82-4d99-a095-d70f50240a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:16.295871 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657aeb9a-ca82-4d99-a095-d70f50240a07 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:16.296158 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657aeb9a-ca82-4d99-a095-d70f50240a07 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:16.296449 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657aeb9a-ca82-4d99-a095-d70f50240a07 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:16.297258 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657aeb9a-ca82-4d99-a095-d70f50240a07 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:16.297570 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657aeb9a-ca82-4d99-a095-d70f50240a07 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:16.298948 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657aeb9a-ca82-4d99-a095-d70f50240a07 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:16.307850 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 433/877] 10.222.0.26 () {56 vars in 1115 bytes} [Sat Aug 6 01:21:16 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:16.313363 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722d519a-54f0-40b0-ac22-1bbe5aa9ab9b None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:16.429176 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722d519a-54f0-40b0-ac22-1bbe5aa9ab9b None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:16.464992 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59aa515-46d7-4dee-96f7-95b74a77a82d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:16.465668 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59aa515-46d7-4dee-96f7-95b74a77a82d None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:16.465953 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59aa515-46d7-4dee-96f7-95b74a77a82d None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:16.466233 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59aa515-46d7-4dee-96f7-95b74a77a82d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:16.477557 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59aa515-46d7-4dee-96f7-95b74a77a82d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:16.479626 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59aa515-46d7-4dee-96f7-95b74a77a82d None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:16.494114 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-a59aa515-46d7-4dee-96f7-95b74a77a82d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:21:16.494114 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:16.494114 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:21:16.494114 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:21:16.494114 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:21:16.494114 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:21:16.494114 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:21:16.494114 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:21:16.494114 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:21:16.494114 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:21:16.494114 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:21:16.494114 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:21:16.494114 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:21:16.494114 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:21:16.494114 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:21:16.495569 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:21:16.495569 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:21:16.495569 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:21:16.495569 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:21:16.495569 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:21:16.495569 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:21:16.495569 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:21:16.495569 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:21:16.495569 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:21:16.495569 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:21:16.495569 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:21:16.495569 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:21:16.495569 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:21:16.495569 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:21:16.495569 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:21:16.495569 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:21:16.495569 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:21:16.496541 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:21:16.496541 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:21:16.496541 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:21:16.496541 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:21:16.496541 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 445/878] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:21:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:21:16.502148 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657aeb9a-ca82-4d99-a095-d70f50240a07 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:16.561998 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:16.609761 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657aeb9a-ca82-4d99-a095-d70f50240a07 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:16.647490 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aea73f8-ff3b-4920-8167-d56e9a17a570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:16.648038 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aea73f8-ff3b-4920-8167-d56e9a17a570 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:16.648324 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aea73f8-ff3b-4920-8167-d56e9a17a570 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:16.648605 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aea73f8-ff3b-4920-8167-d56e9a17a570 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:16.649338 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aea73f8-ff3b-4920-8167-d56e9a17a570 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:16.649647 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aea73f8-ff3b-4920-8167-d56e9a17a570 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:16.651053 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aea73f8-ff3b-4920-8167-d56e9a17a570 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:16.663698 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 434/879] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Aug 6 01:21:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:16.669489 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59aa515-46d7-4dee-96f7-95b74a77a82d None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:16.706994 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:16.777602 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59aa515-46d7-4dee-96f7-95b74a77a82d None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:16.813248 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5da538-112c-4452-b257-2acdc5b96057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:16.813896 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5da538-112c-4452-b257-2acdc5b96057 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:16.814181 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5da538-112c-4452-b257-2acdc5b96057 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:16.814459 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5da538-112c-4452-b257-2acdc5b96057 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:16.822308 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5da538-112c-4452-b257-2acdc5b96057 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:16.823853 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5da538-112c-4452-b257-2acdc5b96057 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:16.835521 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-3f5da538-112c-4452-b257-2acdc5b96057 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 06 01:21:16.835521 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:16.835521 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:21:16.835521 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:21:16.835521 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:21:16.835521 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:21:16.835521 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:21:16.835521 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:21:16.835521 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:21:16.835521 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:21:16.835521 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:21:16.835521 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:21:16.835521 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 01:21:16.835521 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 01:21:16.835521 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 01:21:16.835521 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 01:21:16.837439 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:21:16.837439 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:21:16.837439 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 01:21:16.837439 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:21:16.837439 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 01:21:16.837439 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:21:16.837439 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:21:16.837439 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:21:16.837439 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:21:16.837439 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:21:16.837439 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:21:16.837439 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:21:16.837439 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:21:16.837439 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:21:16.837439 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:21:16.837439 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:21:16.837439 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:21:16.837439 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:21:16.839568 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:21:16.839568 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:21:16.839568 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 01:21:16.839568 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 01:21:16.839568 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 01:21:16.839568 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 01:21:16.839568 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 01:21:16.839568 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 01:21:16.839568 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 06 01:21:16.839568 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:21:16.839568 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 446/880] 10.222.0.26 () {56 vars in 1117 bytes} [Sat Aug 6 01:21:16 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:21:16.843015 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aea73f8-ff3b-4920-8167-d56e9a17a570 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:16.892345 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983ae211-ef79-4701-a93c-478cce660bf1 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:16.892631 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983ae211-ef79-4701-a93c-478cce660bf1 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:16.892915 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983ae211-ef79-4701-a93c-478cce660bf1 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:16.894426 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 447/881] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:21:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:21:16.900774 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb226dc7-c956-47f0-99f7-17f4cd458358 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:16.901401 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb226dc7-c956-47f0-99f7-17f4cd458358 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:16.901781 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb226dc7-c956-47f0-99f7-17f4cd458358 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:16.924025 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-bb226dc7-c956-47f0-99f7-17f4cd458358 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:21:16.925433 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-bb226dc7-c956-47f0-99f7-17f4cd458358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:21:16.969739 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aea73f8-ff3b-4920-8167-d56e9a17a570 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:16.992756 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:17.015280 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 448/882] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:21:16 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 06 01:21:17.017012 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6957c7ce-6d99-4bec-949e-28e064db5eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:17.017876 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6957c7ce-6d99-4bec-949e-28e064db5eb0 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:17.018291 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6957c7ce-6d99-4bec-949e-28e064db5eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:17.018723 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6957c7ce-6d99-4bec-949e-28e064db5eb0 None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:17.019640 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6957c7ce-6d99-4bec-949e-28e064db5eb0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:17.020076 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6957c7ce-6d99-4bec-949e-28e064db5eb0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:17.021683 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6957c7ce-6d99-4bec-949e-28e064db5eb0 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:17.038069 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb226dc7-c956-47f0-99f7-17f4cd458358 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:17.048634 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 435/883] 10.222.0.26 () {56 vars in 1149 bytes} [Sat Aug 6 01:21:16 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:17.054457 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6957c7ce-6d99-4bec-949e-28e064db5eb0 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:17.066394 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:17.147073 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb226dc7-c956-47f0-99f7-17f4cd458358 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:17.180296 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6957c7ce-6d99-4bec-949e-28e064db5eb0 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:17.182433 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a30f7a-fcef-4f7a-900d-3f86a577166e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:17.183069 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a30f7a-fcef-4f7a-900d-3f86a577166e None admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:17.183352 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a30f7a-fcef-4f7a-900d-3f86a577166e None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:17.183638 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a30f7a-fcef-4f7a-900d-3f86a577166e None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:17.184329 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a30f7a-fcef-4f7a-900d-3f86a577166e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:17.185721 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a30f7a-fcef-4f7a-900d-3f86a577166e None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:17.202599 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 449/884] 10.222.0.26 () {60 vars in 1141 bytes} [Sat Aug 6 01:21:17 2022] POST /identity/v3/services => generated 269 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:21:17.204010 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9278a432-3018-47ce-8e1b-a495b13df5df None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:17.204305 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9278a432-3018-47ce-8e1b-a495b13df5df None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:17.204586 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9278a432-3018-47ce-8e1b-a495b13df5df None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:17.209104 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 450/885] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:21:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:21:17.211976 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2f9739-6daa-454a-99d4-ae75b016986a None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:17.212394 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2f9739-6daa-454a-99d4-ae75b016986a None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:17.212743 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2f9739-6daa-454a-99d4-ae75b016986a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:17.238498 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-ff2f9739-6daa-454a-99d4-ae75b016986a None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:21:17.240535 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-ff2f9739-6daa-454a-99d4-ae75b016986a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:21:17.244457 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fa80c2-8c33-4817-af8a-450428ba0676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:17.245837 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fa80c2-8c33-4817-af8a-450428ba0676 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:17.246300 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fa80c2-8c33-4817-af8a-450428ba0676 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:17.246750 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fa80c2-8c33-4817-af8a-450428ba0676 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:17.300581 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:17.335479 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fa80c2-8c33-4817-af8a-450428ba0676 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:17.336961 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fa80c2-8c33-4817-af8a-450428ba0676 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:17.340277 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 451/886] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:21:17 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 06 01:21:17.341447 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3affd534-5030-4852-ba35-58697c0c31c0 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:17.341733 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3affd534-5030-4852-ba35-58697c0c31c0 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:17.342004 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3affd534-5030-4852-ba35-58697c0c31c0 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:17.347320 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 452/887] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:21:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:21:17.350646 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8556e8-ad5e-4815-8066-39eaf0f7675f None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:17.350930 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8556e8-ad5e-4815-8066-39eaf0f7675f None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:17.351462 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8556e8-ad5e-4815-8066-39eaf0f7675f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:17.373222 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-1c8556e8-ad5e-4815-8066-39eaf0f7675f None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:21:17.374418 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-1c8556e8-ad5e-4815-8066-39eaf0f7675f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:21:17.385769 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:17.452118 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 453/888] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:21:17 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 06 01:21:17.453215 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8556e8-ad5e-4815-8066-39eaf0f7675f None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:17.475911 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 436/889] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Aug 6 01:21:17 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 06 01:21:17.477971 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fa80c2-8c33-4817-af8a-450428ba0676 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:17.555220 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:17.564707 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8556e8-ad5e-4815-8066-39eaf0f7675f None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:17.586949 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fa80c2-8c33-4817-af8a-450428ba0676 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:17.600175 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04108ea-b789-4961-9f85-792ab6a732b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:17.601400 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04108ea-b789-4961-9f85-792ab6a732b4 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:17.601861 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04108ea-b789-4961-9f85-792ab6a732b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:17.602323 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04108ea-b789-4961-9f85-792ab6a732b4 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:17.603034 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04108ea-b789-4961-9f85-792ab6a732b4 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:17.605308 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04108ea-b789-4961-9f85-792ab6a732b4 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:17.614538 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-e04108ea-b789-4961-9f85-792ab6a732b4 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 06 01:21:17.614538 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:17.614538 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 01:21:17.614538 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 01:21:17.614538 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 01:21:17.614538 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 01:21:17.614538 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 01:21:17.614538 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 01:21:17.614538 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 06 01:21:17.614538 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:21:17.614538 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 01:21:17.614538 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:21:17.614538 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:17.614538 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:21:17.614538 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:21:17.614538 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:21:17.614538 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:21:17.615518 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:21:17.615518 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:21:17.615518 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:21:17.615518 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:21:17.615518 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:21:17.615518 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:21:17.615518 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 01:21:17.615518 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 01:21:17.615518 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 01:21:17.615518 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 01:21:17.615518 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:21:17.615518 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:21:17.615518 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:21:17.615518 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:21:17.615518 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:21:17.615518 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:21:17.616340 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:21:17.616340 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:21:17.616340 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:21:17.616340 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:21:17.616340 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:21:17.616340 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:21:17.616340 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:21:17.616340 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:21:17.616340 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:21:17.616340 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:21:17.616340 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 01:21:17.616340 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 01:21:17.616340 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 06 01:21:17.616340 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:21:17.616340 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 454/890] 10.222.0.26 () {56 vars in 1126 bytes} [Sat Aug 6 01:21:17 2022] GET /identity/v3/services/object-store => generated 93 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:21:17.617150 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04108ea-b789-4961-9f85-792ab6a732b4 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:17.628730 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317ee8aa-ded2-423b-8eaa-06bdd0cd972e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:17.629422 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317ee8aa-ded2-423b-8eaa-06bdd0cd972e None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:17.629706 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317ee8aa-ded2-423b-8eaa-06bdd0cd972e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:17.629983 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317ee8aa-ded2-423b-8eaa-06bdd0cd972e None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:17.630617 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317ee8aa-ded2-423b-8eaa-06bdd0cd972e None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:17.631163 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:17.631981 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317ee8aa-ded2-423b-8eaa-06bdd0cd972e None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:17.641891 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-317ee8aa-ded2-423b-8eaa-06bdd0cd972e None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 06 01:21:17.641891 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:17.641891 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 01:21:17.641891 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 01:21:17.641891 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 01:21:17.641891 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 01:21:17.641891 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 01:21:17.641891 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 01:21:17.641891 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 06 01:21:17.641891 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:21:17.641891 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 01:21:17.641891 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:21:17.641891 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:17.641891 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:21:17.641891 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:21:17.641891 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:21:17.641891 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:21:17.642922 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:21:17.642922 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:21:17.642922 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:21:17.642922 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:21:17.642922 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:21:17.642922 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:21:17.642922 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 01:21:17.642922 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 01:21:17.642922 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 01:21:17.642922 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 01:21:17.642922 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:21:17.642922 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:21:17.642922 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:21:17.642922 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:21:17.642922 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:21:17.642922 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:21:17.643753 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:21:17.643753 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:21:17.643753 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:21:17.643753 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:21:17.643753 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:21:17.643753 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:21:17.643753 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:21:17.643753 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:21:17.643753 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:21:17.643753 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:21:17.643753 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 01:21:17.643753 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 01:21:17.643753 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 06 01:21:17.643753 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:21:17.643753 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 437/891] 10.222.0.26 () {56 vars in 1114 bytes} [Sat Aug 6 01:21:17 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:21:17.644572 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317ee8aa-ded2-423b-8eaa-06bdd0cd972e None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:17.734614 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04108ea-b789-4961-9f85-792ab6a732b4 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:17.756116 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317ee8aa-ded2-423b-8eaa-06bdd0cd972e None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:17.770722 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65396c59-e256-4730-89bd-c408a2d55e1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:17.771361 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65396c59-e256-4730-89bd-c408a2d55e1c None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:17.771638 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65396c59-e256-4730-89bd-c408a2d55e1c None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:17.771909 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65396c59-e256-4730-89bd-c408a2d55e1c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:17.783810 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65396c59-e256-4730-89bd-c408a2d55e1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:17.785272 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65396c59-e256-4730-89bd-c408a2d55e1c None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:17.792390 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3b5a4b-3eac-44a2-ab7c-999c1510ab2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:17.793008 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3b5a4b-3eac-44a2-ab7c-999c1510ab2d None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:17.793540 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3b5a4b-3eac-44a2-ab7c-999c1510ab2d None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:17.793649 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3b5a4b-3eac-44a2-ab7c-999c1510ab2d None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:17.794370 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3b5a4b-3eac-44a2-ab7c-999c1510ab2d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:17.794600 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3b5a4b-3eac-44a2-ab7c-999c1510ab2d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:17.795958 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3b5a4b-3eac-44a2-ab7c-999c1510ab2d None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:17.797199 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:17.800557 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-65396c59-e256-4730-89bd-c408a2d55e1c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:21:17.800557 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:17.800557 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:21:17.800557 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:21:17.800557 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:21:17.800557 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:21:17.800557 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:21:17.800557 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:21:17.800557 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:21:17.800557 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:21:17.800557 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:21:17.800557 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:21:17.800557 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:21:17.800557 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:21:17.800557 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:21:17.801469 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:21:17.801469 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:21:17.801469 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:21:17.801469 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:21:17.801469 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:21:17.801469 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:21:17.801469 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:21:17.801469 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:21:17.801469 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:21:17.801469 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:21:17.801469 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:21:17.801469 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:21:17.801469 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:21:17.801469 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:21:17.801469 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:21:17.801469 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:21:17.801469 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:21:17.802365 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:21:17.802365 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:21:17.802365 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:21:17.802365 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:21:17.806376 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 455/893] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:21:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:21:17.806488 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 438/893] 10.222.0.26 () {56 vars in 1122 bytes} [Sat Aug 6 01:21:17 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:17.807666 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65396c59-e256-4730-89bd-c408a2d55e1c None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:17.807831 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3b5a4b-3eac-44a2-ab7c-999c1510ab2d None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:17.872393 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:17.916409 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65396c59-e256-4730-89bd-c408a2d55e1c None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:17.916603 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3b5a4b-3eac-44a2-ab7c-999c1510ab2d None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:17.954607 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8effcc-2bdc-4bf5-bf87-188cd169d537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:17.955230 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8effcc-2bdc-4bf5-bf87-188cd169d537 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:17.955515 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8effcc-2bdc-4bf5-bf87-188cd169d537 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:17.955800 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8effcc-2bdc-4bf5-bf87-188cd169d537 None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:17.956486 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8effcc-2bdc-4bf5-bf87-188cd169d537 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:17.956782 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8effcc-2bdc-4bf5-bf87-188cd169d537 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:17.957429 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202871eb-79da-45de-ac40-b045ff9a77df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:17.958220 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202871eb-79da-45de-ac40-b045ff9a77df None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:17.958334 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8effcc-2bdc-4bf5-bf87-188cd169d537 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:17.958742 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202871eb-79da-45de-ac40-b045ff9a77df None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:17.959417 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202871eb-79da-45de-ac40-b045ff9a77df None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:17.960611 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202871eb-79da-45de-ac40-b045ff9a77df None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:17.961096 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202871eb-79da-45de-ac40-b045ff9a77df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:17.962819 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202871eb-79da-45de-ac40-b045ff9a77df None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:17.963083 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 456/894] 10.222.0.26 () {56 vars in 1114 bytes} [Sat Aug 6 01:21:17 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:17.964414 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8effcc-2bdc-4bf5-bf87-188cd169d537 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:17.976290 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 439/895] 10.222.0.26 () {56 vars in 1122 bytes} [Sat Aug 6 01:21:17 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:17.978131 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202871eb-79da-45de-ac40-b045ff9a77df None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:18.072407 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:18.086442 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8effcc-2bdc-4bf5-bf87-188cd169d537 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:18.126559 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:18.127995 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202871eb-79da-45de-ac40-b045ff9a77df None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:18.140759 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9a5168-bc4c-4dd7-85d6-a39c196cef39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:18.140759 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9a5168-bc4c-4dd7-85d6-a39c196cef39 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:18.141068 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9a5168-bc4c-4dd7-85d6-a39c196cef39 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:18.142079 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9a5168-bc4c-4dd7-85d6-a39c196cef39 None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:18.142181 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9a5168-bc4c-4dd7-85d6-a39c196cef39 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:18.142454 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9a5168-bc4c-4dd7-85d6-a39c196cef39 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:18.145415 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9a5168-bc4c-4dd7-85d6-a39c196cef39 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:18.162632 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 457/896] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Aug 6 01:21:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:18.169458 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9a5168-bc4c-4dd7-85d6-a39c196cef39 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:18.172966 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49404666-7119-4cbf-96a4-e3a907bc2fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:18.173773 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49404666-7119-4cbf-96a4-e3a907bc2fa1 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:18.174049 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49404666-7119-4cbf-96a4-e3a907bc2fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:18.174328 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49404666-7119-4cbf-96a4-e3a907bc2fa1 None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:18.175007 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49404666-7119-4cbf-96a4-e3a907bc2fa1 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:18.175309 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49404666-7119-4cbf-96a4-e3a907bc2fa1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:18.176734 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49404666-7119-4cbf-96a4-e3a907bc2fa1 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:18.187471 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 440/897] 10.222.0.26 () {56 vars in 1114 bytes} [Sat Aug 6 01:21:17 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:18.188672 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49404666-7119-4cbf-96a4-e3a907bc2fa1 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:18.287741 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9a5168-bc4c-4dd7-85d6-a39c196cef39 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:18.300864 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49404666-7119-4cbf-96a4-e3a907bc2fa1 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:18.330887 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9360107d-3701-4410-88ba-8bd0daad2b89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:18.331455 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9360107d-3701-4410-88ba-8bd0daad2b89 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:18.331737 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9360107d-3701-4410-88ba-8bd0daad2b89 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:18.332013 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9360107d-3701-4410-88ba-8bd0daad2b89 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:18.332826 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:18.341297 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9360107d-3701-4410-88ba-8bd0daad2b89 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:18.342805 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9360107d-3701-4410-88ba-8bd0daad2b89 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:18.344931 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2072128-09b2-458a-a44d-07e5ba3f0479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:18.346213 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2072128-09b2-458a-a44d-07e5ba3f0479 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:18.346706 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2072128-09b2-458a-a44d-07e5ba3f0479 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:18.347194 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2072128-09b2-458a-a44d-07e5ba3f0479 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:18.348155 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2072128-09b2-458a-a44d-07e5ba3f0479 None admin] RBAC: Adding query filter params (interface=public, service_id=4de1d6d14707443790e4e36d57107676, region_id=RegionOne) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:18.348805 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2072128-09b2-458a-a44d-07e5ba3f0479 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:18.350478 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2072128-09b2-458a-a44d-07e5ba3f0479 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:18.356059 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-9360107d-3701-4410-88ba-8bd0daad2b89 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:21:18.356059 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:18.356059 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:21:18.356059 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:21:18.356059 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:21:18.356059 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:21:18.356059 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:21:18.356059 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:21:18.356059 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:21:18.356059 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:21:18.356059 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:21:18.356059 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:21:18.356059 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 01:21:18.356059 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 01:21:18.356059 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 01:21:18.357001 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 01:21:18.357001 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:21:18.357001 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:21:18.357001 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:21:18.357001 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:21:18.357001 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:21:18.357001 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:21:18.357001 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:21:18.357001 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:21:18.357001 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:21:18.357001 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:21:18.357001 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:21:18.357001 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:21:18.357001 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:21:18.357001 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:21:18.357001 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:21:18.357001 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:21:18.357966 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 01:21:18.357966 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 01:21:18.357966 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 01:21:18.357966 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 01:21:18.357966 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 01:21:18.357966 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 01:21:18.357966 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:21:18.357966 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:21:18.357966 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 458/898] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Aug 6 01:21:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:21:18.360048 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 441/899] 10.222.0.26 () {56 vars in 1251 bytes} [Sat Aug 6 01:21:18 2022] GET /identity/v3/endpoints?service_id=4de1d6d14707443790e4e36d57107676&interface=public®ion_id=RegionOne => generated 194 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:21:18.365180 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9360107d-3701-4410-88ba-8bd0daad2b89 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:18.366422 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2072128-09b2-458a-a44d-07e5ba3f0479 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:18.427801 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:18.484811 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2072128-09b2-458a-a44d-07e5ba3f0479 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:18.496732 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9360107d-3701-4410-88ba-8bd0daad2b89 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:18.525369 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a526bd-4d4b-421f-a975-4b5247d210b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:18.526105 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a526bd-4d4b-421f-a975-4b5247d210b8 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:18.526621 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a526bd-4d4b-421f-a975-4b5247d210b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:18.526971 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a526bd-4d4b-421f-a975-4b5247d210b8 None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:18.527628 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a526bd-4d4b-421f-a975-4b5247d210b8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:18.529146 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a526bd-4d4b-421f-a975-4b5247d210b8 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:18.541239 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 442/900] 10.222.0.26 () {56 vars in 1087 bytes} [Sat Aug 6 01:21:18 2022] GET /identity/v3/services => generated 1268 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:21:18.545313 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bce1e4-36e0-4f4b-9a40-86c907adfd7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:18.546224 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bce1e4-36e0-4f4b-9a40-86c907adfd7f None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:18.546718 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bce1e4-36e0-4f4b-9a40-86c907adfd7f None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:18.547198 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bce1e4-36e0-4f4b-9a40-86c907adfd7f None admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:18.548221 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bce1e4-36e0-4f4b-9a40-86c907adfd7f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:18.548716 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bce1e4-36e0-4f4b-9a40-86c907adfd7f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:18.550486 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bce1e4-36e0-4f4b-9a40-86c907adfd7f None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:18.565621 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 459/901] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Aug 6 01:21:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:18.572284 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a526bd-4d4b-421f-a975-4b5247d210b8 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:18.648480 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:18.699360 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a526bd-4d4b-421f-a975-4b5247d210b8 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:18.739123 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f73cdc-f5f9-42ee-9e1f-b2a8f9c6bf57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:18.739779 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f73cdc-f5f9-42ee-9e1f-b2a8f9c6bf57 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:18.740094 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f73cdc-f5f9-42ee-9e1f-b2a8f9c6bf57 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:18.740377 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f73cdc-f5f9-42ee-9e1f-b2a8f9c6bf57 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:18.741282 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f73cdc-f5f9-42ee-9e1f-b2a8f9c6bf57 None admin] RBAC: Adding query filter params (role.id=f42c599286be4e0fa632dc6769f95753, scope.project.id=d9d68d310bf24449a5fe20d2dc264a58, user.id=8e86f38590824894a3b960e6774ff8a9) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:18.741601 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f73cdc-f5f9-42ee-9e1f-b2a8f9c6bf57 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:18.743133 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f73cdc-f5f9-42ee-9e1f-b2a8f9c6bf57 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:18.750326 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 443/902] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Aug 6 01:21:18 2022] GET /identity/v3/role_assignments?user.id=8e86f38590824894a3b960e6774ff8a9&scope.project.id=d9d68d310bf24449a5fe20d2dc264a58&role.id=f42c599286be4e0fa632dc6769f95753 => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:19.298097 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd546c8d-e7cf-4d0d-b708-99d061b7a683 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:19.298615 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd546c8d-e7cf-4d0d-b708-99d061b7a683 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:19.298615 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd546c8d-e7cf-4d0d-b708-99d061b7a683 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:19.300205 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 460/903] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:21:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:21:19.305577 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a186cac7-9f42-4778-8ff3-27eda3697a0f None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:19.305956 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a186cac7-9f42-4778-8ff3-27eda3697a0f None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:19.306148 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a186cac7-9f42-4778-8ff3-27eda3697a0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:19.325113 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-a186cac7-9f42-4778-8ff3-27eda3697a0f None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:21:19.326304 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-a186cac7-9f42-4778-8ff3-27eda3697a0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:21:19.400077 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 444/904] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:21:19 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:21:19.423250 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd546c8d-e7cf-4d0d-b708-99d061b7a683 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:19.530912 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd546c8d-e7cf-4d0d-b708-99d061b7a683 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:19.538427 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:19.566522 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3937fed-7c23-4bfe-a0cb-20b033a7efcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:19.567094 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3937fed-7c23-4bfe-a0cb-20b033a7efcb None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:19.567381 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3937fed-7c23-4bfe-a0cb-20b033a7efcb None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:19.567662 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3937fed-7c23-4bfe-a0cb-20b033a7efcb None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:19.568293 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3937fed-7c23-4bfe-a0cb-20b033a7efcb None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:19.569741 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3937fed-7c23-4bfe-a0cb-20b033a7efcb None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:19.579577 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-b3937fed-7c23-4bfe-a0cb-20b033a7efcb None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 06 01:21:19.579577 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:19.579577 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 01:21:19.579577 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 01:21:19.579577 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 01:21:19.579577 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 01:21:19.579577 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 01:21:19.579577 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 01:21:19.579577 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 06 01:21:19.579577 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:21:19.579577 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 01:21:19.579577 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:21:19.579577 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:19.579577 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:21:19.579577 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:21:19.579577 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:21:19.580481 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:21:19.580481 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:21:19.580481 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:21:19.580481 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:21:19.580481 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:21:19.580481 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:21:19.580481 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:21:19.580481 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 01:21:19.580481 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 01:21:19.580481 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 01:21:19.580481 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 01:21:19.580481 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:21:19.580481 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:21:19.580481 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:21:19.580481 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:21:19.580481 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:21:19.581535 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:21:19.581535 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:21:19.581535 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:21:19.581535 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:21:19.581535 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:21:19.581535 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:21:19.581535 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:21:19.581535 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:21:19.581535 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:21:19.581535 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:21:19.581535 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:21:19.581535 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 01:21:19.581535 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 01:21:19.581535 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 06 01:21:19.581535 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:21:19.581535 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 461/905] 10.222.0.26 () {56 vars in 1132 bytes} [Sat Aug 6 01:21:19 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:21:19.586880 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a186cac7-9f42-4778-8ff3-27eda3697a0f None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:19.638201 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:19.693225 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a186cac7-9f42-4778-8ff3-27eda3697a0f None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:19.730167 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd08e778-68c8-441a-b782-be7b595662fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:19.731092 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd08e778-68c8-441a-b782-be7b595662fd None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:19.731559 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd08e778-68c8-441a-b782-be7b595662fd None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:19.732042 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd08e778-68c8-441a-b782-be7b595662fd None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:19.732912 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd08e778-68c8-441a-b782-be7b595662fd None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:19.733424 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd08e778-68c8-441a-b782-be7b595662fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:19.735037 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd08e778-68c8-441a-b782-be7b595662fd None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:19.740743 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 445/906] 10.222.0.26 () {56 vars in 1126 bytes} [Sat Aug 6 01:21:19 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:19.746713 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3937fed-7c23-4bfe-a0cb-20b033a7efcb None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:19.855734 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3937fed-7c23-4bfe-a0cb-20b033a7efcb None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:19.893418 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76e720f-eab2-4141-81a4-04073faf31f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:19.894042 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76e720f-eab2-4141-81a4-04073faf31f8 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:19.894328 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76e720f-eab2-4141-81a4-04073faf31f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:19.894602 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76e720f-eab2-4141-81a4-04073faf31f8 None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:19.895279 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76e720f-eab2-4141-81a4-04073faf31f8 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:19.895584 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76e720f-eab2-4141-81a4-04073faf31f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:19.896948 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76e720f-eab2-4141-81a4-04073faf31f8 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:19.902824 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 462/907] 10.222.0.26 () {56 vars in 1126 bytes} [Sat Aug 6 01:21:19 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:19.908676 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd08e778-68c8-441a-b782-be7b595662fd None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:20.012323 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd08e778-68c8-441a-b782-be7b595662fd None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:20.026648 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:20.054901 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471dd1ad-efcf-4b86-9d3b-23dc2865140d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:20.055787 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471dd1ad-efcf-4b86-9d3b-23dc2865140d None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:20.056278 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471dd1ad-efcf-4b86-9d3b-23dc2865140d None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:20.056739 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471dd1ad-efcf-4b86-9d3b-23dc2865140d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:20.057735 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471dd1ad-efcf-4b86-9d3b-23dc2865140d None admin] RBAC: Adding query filter params (interface=public, service_id=4abe152231174bc886ad943bbe1881e9, region_id=RegionOne) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:20.058227 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471dd1ad-efcf-4b86-9d3b-23dc2865140d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:20.059760 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471dd1ad-efcf-4b86-9d3b-23dc2865140d None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:20.065680 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 446/908] 10.222.0.26 () {56 vars in 1251 bytes} [Sat Aug 6 01:21:19 2022] GET /identity/v3/endpoints?service_id=4abe152231174bc886ad943bbe1881e9&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:20.071806 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76e720f-eab2-4141-81a4-04073faf31f8 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:20.111016 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:20.152197 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a1f017-fed5-4eda-ba17-97080b69cb1d None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:20.152576 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a1f017-fed5-4eda-ba17-97080b69cb1d None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:20.152933 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a1f017-fed5-4eda-ba17-97080b69cb1d None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:20.154664 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 447/909] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:21:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 01:21:20.159571 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02018273-d814-4b48-977d-6b872464003e None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:20.159930 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02018273-d814-4b48-977d-6b872464003e None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:20.160289 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02018273-d814-4b48-977d-6b872464003e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:20.178346 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-02018273-d814-4b48-977d-6b872464003e None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:21:20.179645 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-02018273-d814-4b48-977d-6b872464003e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:21:20.181897 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76e720f-eab2-4141-81a4-04073faf31f8 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:20.217554 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e417235-86e2-430a-815b-3a148b5141a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:20.218171 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e417235-86e2-430a-815b-3a148b5141a6 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:20.218451 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e417235-86e2-430a-815b-3a148b5141a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:20.218725 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e417235-86e2-430a-815b-3a148b5141a6 None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:20.219374 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e417235-86e2-430a-815b-3a148b5141a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:20.220966 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e417235-86e2-430a-815b-3a148b5141a6 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:20.227361 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 463/910] 10.222.0.26 () {56 vars in 1087 bytes} [Sat Aug 6 01:21:20 2022] GET /identity/v3/services => generated 1268 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:21:20.258348 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 448/911] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:21:20 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 06 01:21:20.284171 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e417235-86e2-430a-815b-3a148b5141a6 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:20.398967 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:20.405344 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e417235-86e2-430a-815b-3a148b5141a6 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:20.442626 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b895df-deee-4878-89cb-2d9bb1d4423c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:20.443264 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b895df-deee-4878-89cb-2d9bb1d4423c None admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:20.443551 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b895df-deee-4878-89cb-2d9bb1d4423c None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:20.443827 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b895df-deee-4878-89cb-2d9bb1d4423c None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:20.444536 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b895df-deee-4878-89cb-2d9bb1d4423c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:20.445968 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b895df-deee-4878-89cb-2d9bb1d4423c None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:20.468619 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 464/912] 10.222.0.26 () {60 vars in 1141 bytes} [Sat Aug 6 01:21:20 2022] POST /identity/v3/services => generated 241 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:21:20.774071 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ad4bff-cde7-41ca-ad7b-a7e0ad8acd44 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:20.774586 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ad4bff-cde7-41ca-ad7b-a7e0ad8acd44 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:20.774661 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ad4bff-cde7-41ca-ad7b-a7e0ad8acd44 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:20.776446 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 449/913] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:21:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:21:20.781662 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1d53f3-12ba-45f3-a99a-02c9c73e3be2 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:20.781954 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1d53f3-12ba-45f3-a99a-02c9c73e3be2 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:20.782236 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1d53f3-12ba-45f3-a99a-02c9c73e3be2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:20.795034 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bd621d-23ce-444b-90ce-95d3c59d3470 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:20.795352 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bd621d-23ce-444b-90ce-95d3c59d3470 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:20.795585 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bd621d-23ce-444b-90ce-95d3c59d3470 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:20.797192 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 450/914] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:21:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 01:21:20.801681 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-3b1d53f3-12ba-45f3-a99a-02c9c73e3be2 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:21:20.802101 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c1d2fe-8541-469f-b61c-571be5f9c346 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:20.802389 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c1d2fe-8541-469f-b61c-571be5f9c346 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:20.802668 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c1d2fe-8541-469f-b61c-571be5f9c346 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:20.802874 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-3b1d53f3-12ba-45f3-a99a-02c9c73e3be2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:21:20.821502 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-69c1d2fe-8541-469f-b61c-571be5f9c346 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:21:20.823327 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-69c1d2fe-8541-469f-b61c-571be5f9c346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:21:20.897623 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 465/915] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:21:20 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:21:20.902917 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 451/916] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:21:20 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 06 01:21:20.920811 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1d53f3-12ba-45f3-a99a-02c9c73e3be2 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:20.929863 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c1d2fe-8541-469f-b61c-571be5f9c346 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:20.981708 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:21.021341 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:21.044218 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1d53f3-12ba-45f3-a99a-02c9c73e3be2 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:21.067355 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c1d2fe-8541-469f-b61c-571be5f9c346 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:21.084299 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fc635d-7f6c-4136-8324-377c301ec800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:21.085393 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fc635d-7f6c-4136-8324-377c301ec800 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:21.085929 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fc635d-7f6c-4136-8324-377c301ec800 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:21.086399 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fc635d-7f6c-4136-8324-377c301ec800 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:21.087227 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fc635d-7f6c-4136-8324-377c301ec800 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:21.089618 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fc635d-7f6c-4136-8324-377c301ec800 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:21.099519 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-34fc635d-7f6c-4136-8324-377c301ec800 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 06 01:21:21.099519 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:21.099519 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 01:21:21.099519 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 01:21:21.099519 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 01:21:21.099519 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 01:21:21.099519 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 01:21:21.099519 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 01:21:21.099519 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 06 01:21:21.099519 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:21:21.099519 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 01:21:21.099519 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:21:21.099519 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:21.099519 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:21:21.099519 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:21:21.099519 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:21:21.099519 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:21:21.100509 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:21:21.100509 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:21:21.100509 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:21:21.100509 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:21:21.100509 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:21:21.100509 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:21:21.100509 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 01:21:21.100509 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 01:21:21.100509 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 01:21:21.100509 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 01:21:21.100509 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:21:21.100509 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:21:21.100509 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:21:21.100509 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:21:21.100509 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:21:21.100509 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:21:21.102007 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:21:21.102007 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:21:21.102007 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:21:21.102007 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:21:21.102007 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:21:21.102007 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:21:21.102007 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:21:21.102007 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:21:21.102007 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:21:21.102007 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:21:21.102007 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 01:21:21.102007 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 01:21:21.102007 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 06 01:21:21.102007 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:21:21.109575 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ac496d-4000-407c-8826-b817aaba56a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:21.109575 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ac496d-4000-407c-8826-b817aaba56a6 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:21.109575 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ac496d-4000-407c-8826-b817aaba56a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:21.109575 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ac496d-4000-407c-8826-b817aaba56a6 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:21.110485 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 466/917] 10.222.0.26 () {56 vars in 1114 bytes} [Sat Aug 6 01:21:20 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:21:21.112367 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fc635d-7f6c-4136-8324-377c301ec800 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:21.121817 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ac496d-4000-407c-8826-b817aaba56a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:21:21.127666 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-c8ac496d-4000-407c-8826-b817aaba56a6 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 06 01:21:21.127666 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:21.127666 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:21:21.127666 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:21:21.127666 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:21:21.127666 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:21:21.127666 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:21:21.127666 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:21:21.127666 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:21:21.127666 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:21:21.127666 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:21:21.127666 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:21:21.127666 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:21:21.127666 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:21:21.127666 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:21:21.127666 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:21:21.128589 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:21:21.128589 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:21:21.128589 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:21:21.128589 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:21:21.128589 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:21:21.128589 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:21:21.128589 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:21:21.128589 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:21:21.128589 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:21:21.128589 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:21:21.128589 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:21:21.128589 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:21:21.128589 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:21:21.128589 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:21:21.128589 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:21:21.128589 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:21:21.129456 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:21:21.129456 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:21:21.129456 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:21:21.129456 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:21:21.129456 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:21:21.129456 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:21:21.129456 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:21:21.129456 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:21:21.129456 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:21:21.129456 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:21:21.129456 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 06 01:21:21.129456 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:21:21.133149 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 452/918] 10.222.0.26 () {56 vars in 1120 bytes} [Sat Aug 6 01:21:20 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:21:21.135705 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ac496d-4000-407c-8826-b817aaba56a6 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:21.241667 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fc635d-7f6c-4136-8324-377c301ec800 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:21.260509 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:21.268425 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ac496d-4000-407c-8826-b817aaba56a6 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:21.284565 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f044039-5924-4669-8f25-3fe9c1b399ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:21.285789 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f044039-5924-4669-8f25-3fe9c1b399ff None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:21.286311 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f044039-5924-4669-8f25-3fe9c1b399ff None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:21.286956 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f044039-5924-4669-8f25-3fe9c1b399ff None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:21.288133 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f044039-5924-4669-8f25-3fe9c1b399ff None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:21.288646 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f044039-5924-4669-8f25-3fe9c1b399ff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:21.290660 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f044039-5924-4669-8f25-3fe9c1b399ff None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:21.298181 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 467/919] 10.222.0.26 () {56 vars in 1114 bytes} [Sat Aug 6 01:21:21 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:21.303736 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f044039-5924-4669-8f25-3fe9c1b399ff None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:21.312030 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9d1c2c-2c2b-4470-a036-5e220b9da11b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:21.312659 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9d1c2c-2c2b-4470-a036-5e220b9da11b None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:21.312943 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9d1c2c-2c2b-4470-a036-5e220b9da11b None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:21.313255 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9d1c2c-2c2b-4470-a036-5e220b9da11b None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:21.314276 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9d1c2c-2c2b-4470-a036-5e220b9da11b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:21.314831 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9d1c2c-2c2b-4470-a036-5e220b9da11b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:21.317315 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9d1c2c-2c2b-4470-a036-5e220b9da11b None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:21.322508 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:21.333211 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 453/920] 10.222.0.26 () {56 vars in 1115 bytes} [Sat Aug 6 01:21:21 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:21.334091 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9d1c2c-2c2b-4470-a036-5e220b9da11b None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:21.441521 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f044039-5924-4669-8f25-3fe9c1b399ff None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:21.454581 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9d1c2c-2c2b-4470-a036-5e220b9da11b None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:21.485800 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc8ed95-0e34-4a9e-a702-4f6e30d38a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:21.486793 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc8ed95-0e34-4a9e-a702-4f6e30d38a2b None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:21.487223 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc8ed95-0e34-4a9e-a702-4f6e30d38a2b None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:21.487697 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc8ed95-0e34-4a9e-a702-4f6e30d38a2b None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:21.488553 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc8ed95-0e34-4a9e-a702-4f6e30d38a2b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:21.489066 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc8ed95-0e34-4a9e-a702-4f6e30d38a2b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:21.490607 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc8ed95-0e34-4a9e-a702-4f6e30d38a2b None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:21.491791 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421e93a3-bc74-4b65-b790-9b649be21c85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:21.492500 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421e93a3-bc74-4b65-b790-9b649be21c85 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:21.492849 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421e93a3-bc74-4b65-b790-9b649be21c85 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:21.493226 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421e93a3-bc74-4b65-b790-9b649be21c85 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:21.501818 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 468/921] 10.222.0.26 () {56 vars in 1114 bytes} [Sat Aug 6 01:21:21 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:21.502890 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc8ed95-0e34-4a9e-a702-4f6e30d38a2b None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:21.505287 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421e93a3-bc74-4b65-b790-9b649be21c85 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:21.506802 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421e93a3-bc74-4b65-b790-9b649be21c85 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:21.523685 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-421e93a3-bc74-4b65-b790-9b649be21c85 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:21:21.523685 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:21.523685 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:21:21.523685 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:21:21.523685 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:21:21.523685 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:21:21.523685 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:21:21.523685 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:21:21.523685 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:21:21.523685 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:21:21.523685 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:21:21.523685 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:21:21.523685 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:21:21.523685 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:21:21.523685 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:21:21.524604 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:21:21.524604 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:21:21.524604 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:21:21.524604 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:21:21.524604 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:21:21.524604 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:21:21.524604 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:21:21.524604 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:21:21.524604 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:21:21.524604 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:21:21.524604 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:21:21.524604 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:21:21.524604 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:21:21.524604 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:21:21.524604 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:21:21.524604 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:21:21.524604 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:21:21.526154 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:21:21.526154 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:21:21.526154 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:21:21.526154 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:21:21.529328 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 454/922] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:21:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:21:21.531821 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421e93a3-bc74-4b65-b790-9b649be21c85 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:21.545459 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:21.597920 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:21.626091 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc8ed95-0e34-4a9e-a702-4f6e30d38a2b None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:21.660110 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421e93a3-bc74-4b65-b790-9b649be21c85 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:21.663331 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cff40d-a129-4070-93b8-9bfa4d0c3f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:21.663532 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cff40d-a129-4070-93b8-9bfa4d0c3f79 None admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:21.663882 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cff40d-a129-4070-93b8-9bfa4d0c3f79 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:21.664219 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cff40d-a129-4070-93b8-9bfa4d0c3f79 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:21.665000 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cff40d-a129-4070-93b8-9bfa4d0c3f79 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:21.667527 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cff40d-a129-4070-93b8-9bfa4d0c3f79 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:21.695089 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 469/923] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:21:21 2022] POST /identity/v3/endpoints => generated 346 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:21:21.714155 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaa8225-d6e4-457b-93c7-a022011923ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:21.715175 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaa8225-d6e4-457b-93c7-a022011923ad None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:21.715527 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaa8225-d6e4-457b-93c7-a022011923ad None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:21.715873 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaa8225-d6e4-457b-93c7-a022011923ad None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:21.716661 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eaa8225-d6e4-457b-93c7-a022011923ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:21.717039 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eaa8225-d6e4-457b-93c7-a022011923ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:21.718733 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eaa8225-d6e4-457b-93c7-a022011923ad None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:21.739553 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 455/924] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Aug 6 01:21:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:21:21.741217 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cff40d-a129-4070-93b8-9bfa4d0c3f79 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:21.857865 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cff40d-a129-4070-93b8-9bfa4d0c3f79 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:21.890241 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:21.896194 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2beb4f-0f14-4e09-9911-e0e5a85c278e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:21.897133 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2beb4f-0f14-4e09-9911-e0e5a85c278e None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:21.897622 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2beb4f-0f14-4e09-9911-e0e5a85c278e None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:21.898098 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2beb4f-0f14-4e09-9911-e0e5a85c278e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:21.907224 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2beb4f-0f14-4e09-9911-e0e5a85c278e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:21.908883 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2beb4f-0f14-4e09-9911-e0e5a85c278e None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:21.920927 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-6c2beb4f-0f14-4e09-9911-e0e5a85c278e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 06 01:21:21.920927 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:21.920927 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:21:21.920927 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:21:21.920927 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:21:21.920927 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:21:21.920927 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:21:21.920927 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:21:21.920927 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:21:21.920927 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:21:21.920927 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:21:21.920927 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:21:21.920927 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 01:21:21.920927 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 01:21:21.920927 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 01:21:21.920927 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 01:21:21.922088 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:21:21.922088 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:21:21.922088 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 01:21:21.922088 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:21:21.922088 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 01:21:21.922088 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:21:21.922088 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:21:21.922088 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:21:21.922088 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:21:21.922088 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:21:21.922088 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:21:21.922088 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:21:21.922088 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:21:21.922088 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:21:21.922088 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:21:21.922088 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:21:21.922088 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:21:21.922088 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:21:21.923229 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:21:21.923229 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:21:21.923229 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 01:21:21.923229 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 01:21:21.923229 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 01:21:21.923229 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 01:21:21.923229 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 01:21:21.923229 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 01:21:21.923229 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 06 01:21:21.923229 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:21:21.925424 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 470/925] 10.222.0.26 () {56 vars in 1117 bytes} [Sat Aug 6 01:21:21 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:21:21.930837 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaa8225-d6e4-457b-93c7-a022011923ad None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:22.003438 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:22.040796 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaa8225-d6e4-457b-93c7-a022011923ad None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:22.077607 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9096c55f-0c01-454b-9b31-e4407c0c73b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:22.078239 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9096c55f-0c01-454b-9b31-e4407c0c73b7 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:22.078520 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9096c55f-0c01-454b-9b31-e4407c0c73b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:22.078849 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9096c55f-0c01-454b-9b31-e4407c0c73b7 None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:22.079650 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9096c55f-0c01-454b-9b31-e4407c0c73b7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:22.080014 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9096c55f-0c01-454b-9b31-e4407c0c73b7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:22.081491 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9096c55f-0c01-454b-9b31-e4407c0c73b7 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:22.100275 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 456/926] 10.222.0.26 () {56 vars in 1149 bytes} [Sat Aug 6 01:21:21 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:21:22.106611 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2beb4f-0f14-4e09-9911-e0e5a85c278e None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:22.214886 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2beb4f-0f14-4e09-9911-e0e5a85c278e None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:22.251547 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afae656d-628e-4eb4-b1b7-7079edb3032e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:22.252192 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afae656d-628e-4eb4-b1b7-7079edb3032e None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:22.252472 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afae656d-628e-4eb4-b1b7-7079edb3032e None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:22.252748 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afae656d-628e-4eb4-b1b7-7079edb3032e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:22.263930 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afae656d-628e-4eb4-b1b7-7079edb3032e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:22.265479 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afae656d-628e-4eb4-b1b7-7079edb3032e None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:22.280946 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-afae656d-628e-4eb4-b1b7-7079edb3032e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:21:22.280946 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:22.280946 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:21:22.280946 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:21:22.280946 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:21:22.280946 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:21:22.280946 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:21:22.280946 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:21:22.280946 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:21:22.280946 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:21:22.280946 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:21:22.280946 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:21:22.280946 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:21:22.280946 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:21:22.280946 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:21:22.281930 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:21:22.281930 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:21:22.281930 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:21:22.281930 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:21:22.281930 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:21:22.281930 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:21:22.281930 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:21:22.281930 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:21:22.281930 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:21:22.281930 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:21:22.281930 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:21:22.281930 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:21:22.281930 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:21:22.281930 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:21:22.281930 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:21:22.281930 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:21:22.281930 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:21:22.283101 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:21:22.283101 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:21:22.283101 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:21:22.283101 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:21:22.283101 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 471/927] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:21:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:21:22.289373 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9096c55f-0c01-454b-9b31-e4407c0c73b7 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:22.368393 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bce2ef0-b188-491e-89b5-51e5a7bf0bf2 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:22.369177 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bce2ef0-b188-491e-89b5-51e5a7bf0bf2 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:22.369177 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bce2ef0-b188-491e-89b5-51e5a7bf0bf2 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:22.370766 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 472/928] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:21:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 01:21:22.375654 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101a53eb-8ddf-47de-9611-fa67e95f6455 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:22.375949 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101a53eb-8ddf-47de-9611-fa67e95f6455 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:22.376230 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101a53eb-8ddf-47de-9611-fa67e95f6455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:22.401791 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9096c55f-0c01-454b-9b31-e4407c0c73b7 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:22.409665 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-101a53eb-8ddf-47de-9611-fa67e95f6455 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:21:22.410883 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-101a53eb-8ddf-47de-9611-fa67e95f6455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:21:22.434012 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:22.451386 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d4f5c4-af74-4a07-ae56-b73ba44632df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:22.452601 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d4f5c4-af74-4a07-ae56-b73ba44632df None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:22.453128 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d4f5c4-af74-4a07-ae56-b73ba44632df None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:22.453621 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d4f5c4-af74-4a07-ae56-b73ba44632df None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:22.454627 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d4f5c4-af74-4a07-ae56-b73ba44632df None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:22.455162 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d4f5c4-af74-4a07-ae56-b73ba44632df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:22.457944 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d4f5c4-af74-4a07-ae56-b73ba44632df None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:22.463998 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:22.476485 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 457/929] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Aug 6 01:21:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:22.485107 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d4f5c4-af74-4a07-ae56-b73ba44632df None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:22.502906 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 473/930] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:21:22 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 06 01:21:22.530112 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101a53eb-8ddf-47de-9611-fa67e95f6455 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:22.594174 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d4f5c4-af74-4a07-ae56-b73ba44632df None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:22.630867 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fab0651-c44d-4fa6-85d3-3578637daa8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:22.631545 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fab0651-c44d-4fa6-85d3-3578637daa8f None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:22.631833 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fab0651-c44d-4fa6-85d3-3578637daa8f None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:22.632113 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fab0651-c44d-4fa6-85d3-3578637daa8f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:22.632641 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101a53eb-8ddf-47de-9611-fa67e95f6455 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:22.640421 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fab0651-c44d-4fa6-85d3-3578637daa8f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:22.641978 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fab0651-c44d-4fa6-85d3-3578637daa8f None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:22.654605 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-3fab0651-c44d-4fa6-85d3-3578637daa8f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:21:22.654605 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:22.654605 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:21:22.654605 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:21:22.654605 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:21:22.654605 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:21:22.654605 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:21:22.654605 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:21:22.654605 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:21:22.654605 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:21:22.654605 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:21:22.654605 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:21:22.654605 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 01:21:22.654605 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 01:21:22.654605 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 01:21:22.655482 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 01:21:22.655482 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:21:22.655482 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:21:22.655482 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:21:22.655482 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:21:22.655482 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:21:22.655482 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:21:22.655482 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:21:22.655482 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:21:22.655482 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:21:22.655482 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:21:22.655482 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:21:22.655482 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:21:22.655482 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:21:22.655482 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:21:22.655482 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:21:22.655482 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:21:22.656376 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 01:21:22.656376 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 01:21:22.656376 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 01:21:22.656376 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 01:21:22.656376 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 01:21:22.656376 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 01:21:22.656376 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:21:22.656376 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:21:22.660359 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 458/931] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Aug 6 01:21:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:21:22.661732 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b881a10c-39ed-4d3d-b857-3a072ec88ad4 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:22.662020 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b881a10c-39ed-4d3d-b857-3a072ec88ad4 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:22.662298 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b881a10c-39ed-4d3d-b857-3a072ec88ad4 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:22.663765 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 459/932] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:21:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:21:22.665090 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b881a10c-39ed-4d3d-b857-3a072ec88ad4 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:22.668768 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ec6dc6-9b30-4b08-914a-bc0d1009ae19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:22.669644 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ec6dc6-9b30-4b08-914a-bc0d1009ae19 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:22.669929 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ec6dc6-9b30-4b08-914a-bc0d1009ae19 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:22.670201 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ec6dc6-9b30-4b08-914a-bc0d1009ae19 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:22.670824 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ec6dc6-9b30-4b08-914a-bc0d1009ae19 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:22.672187 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ec6dc6-9b30-4b08-914a-bc0d1009ae19 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:22.681202 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-06ec6dc6-9b30-4b08-914a-bc0d1009ae19 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 06 01:21:22.681202 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:22.681202 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 01:21:22.681202 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 01:21:22.681202 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 01:21:22.681202 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 01:21:22.681202 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 01:21:22.681202 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 01:21:22.681202 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 06 01:21:22.681202 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:21:22.681202 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 01:21:22.681202 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:21:22.681202 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:22.681202 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:21:22.681202 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:21:22.681202 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:21:22.682422 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:21:22.682422 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:21:22.682422 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:21:22.682422 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:21:22.682422 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:21:22.682422 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:21:22.682422 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:21:22.682422 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 01:21:22.682422 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 01:21:22.682422 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 01:21:22.682422 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 01:21:22.682422 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:21:22.682422 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:21:22.682422 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:21:22.682422 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:21:22.682422 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:21:22.683474 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:21:22.683474 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:21:22.683474 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:21:22.683474 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:21:22.683474 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:21:22.683474 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:21:22.683474 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:21:22.683474 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:21:22.683474 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:21:22.683474 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:21:22.683474 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:21:22.683474 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 01:21:22.683474 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 01:21:22.683474 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 06 01:21:22.683474 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:21:22.683474 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 474/933] 10.222.0.26 () {56 vars in 1132 bytes} [Sat Aug 6 01:21:22 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:21:22.684409 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1a2ea3-96c4-425c-ba4c-9ee234f9db48 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:22.684409 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1a2ea3-96c4-425c-ba4c-9ee234f9db48 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:22.684409 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1a2ea3-96c4-425c-ba4c-9ee234f9db48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:22.703255 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-6f1a2ea3-96c4-425c-ba4c-9ee234f9db48 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:21:22.704437 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-6f1a2ea3-96c4-425c-ba4c-9ee234f9db48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:21:22.720060 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:22.734962 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:22.777098 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 475/934] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:21:22 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:21:22.777098 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1a2ea3-96c4-425c-ba4c-9ee234f9db48 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:22.790400 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b881a10c-39ed-4d3d-b857-3a072ec88ad4 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:22.830082 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19998eeb-2f00-402b-9f03-16df2f61fa6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:22.830546 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19998eeb-2f00-402b-9f03-16df2f61fa6b None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:22.830720 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19998eeb-2f00-402b-9f03-16df2f61fa6b None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:22.830965 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19998eeb-2f00-402b-9f03-16df2f61fa6b None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:22.831862 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19998eeb-2f00-402b-9f03-16df2f61fa6b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:22.832244 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19998eeb-2f00-402b-9f03-16df2f61fa6b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:22.834390 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19998eeb-2f00-402b-9f03-16df2f61fa6b None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:22.848626 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 460/935] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Aug 6 01:21:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:22.849475 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19998eeb-2f00-402b-9f03-16df2f61fa6b None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:22.897948 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1a2ea3-96c4-425c-ba4c-9ee234f9db48 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:22.934504 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f62e11f-4060-4046-beb0-003a61b2d99e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:22.935368 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f62e11f-4060-4046-beb0-003a61b2d99e None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:22.935800 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f62e11f-4060-4046-beb0-003a61b2d99e None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:22.936097 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f62e11f-4060-4046-beb0-003a61b2d99e None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:22.936782 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f62e11f-4060-4046-beb0-003a61b2d99e None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:22.937204 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f62e11f-4060-4046-beb0-003a61b2d99e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:22.938933 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f62e11f-4060-4046-beb0-003a61b2d99e None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:22.944333 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 476/936] 10.222.0.26 () {56 vars in 1126 bytes} [Sat Aug 6 01:21:22 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:22.950556 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f62e11f-4060-4046-beb0-003a61b2d99e None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:22.958095 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19998eeb-2f00-402b-9f03-16df2f61fa6b None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:22.960845 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:22.983824 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:22.997540 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea126ec-db0d-4500-a0a1-6d94d4afed0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:22.998126 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea126ec-db0d-4500-a0a1-6d94d4afed0e None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:22.998389 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea126ec-db0d-4500-a0a1-6d94d4afed0e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:22.998663 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea126ec-db0d-4500-a0a1-6d94d4afed0e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:22.999286 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea126ec-db0d-4500-a0a1-6d94d4afed0e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:23.000647 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea126ec-db0d-4500-a0a1-6d94d4afed0e None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:23.009448 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-6ea126ec-db0d-4500-a0a1-6d94d4afed0e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 06 01:21:23.009448 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:23.009448 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 01:21:23.009448 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 01:21:23.009448 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 01:21:23.009448 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 01:21:23.009448 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 01:21:23.009448 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 01:21:23.009448 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 06 01:21:23.009448 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:21:23.009448 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 01:21:23.009448 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:21:23.009448 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:23.009448 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:21:23.009448 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:21:23.009448 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:21:23.009448 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:21:23.010567 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:21:23.010567 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:21:23.010567 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:21:23.010567 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:21:23.010567 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:21:23.010567 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:21:23.010567 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 01:21:23.010567 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 01:21:23.010567 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 01:21:23.010567 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 01:21:23.010567 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:21:23.010567 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:21:23.010567 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:21:23.010567 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:21:23.010567 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:21:23.010567 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:21:23.011496 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:21:23.011496 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:21:23.011496 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:21:23.011496 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:21:23.011496 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:21:23.011496 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:21:23.011496 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:21:23.011496 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:21:23.011496 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:21:23.011496 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:21:23.011496 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 01:21:23.011496 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 01:21:23.011496 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 06 01:21:23.011496 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:21:23.011496 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 461/937] 10.222.0.26 () {56 vars in 1126 bytes} [Sat Aug 6 01:21:22 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:21:23.012348 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea126ec-db0d-4500-a0a1-6d94d4afed0e None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:23.065333 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f62e11f-4060-4046-beb0-003a61b2d99e None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:23.107615 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c1b453-5c84-45f9-95f2-b7fb545a3a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:23.108320 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c1b453-5c84-45f9-95f2-b7fb545a3a5e None admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:23.108600 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c1b453-5c84-45f9-95f2-b7fb545a3a5e None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:23.108868 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c1b453-5c84-45f9-95f2-b7fb545a3a5e None admin] PATH_INFO: `/v3/projects/d9d68d310bf24449a5fe20d2dc264a58/users/8e86f38590824894a3b960e6774ff8a9/roles/f42c599286be4e0fa632dc6769f95753` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:23.111107 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c1b453-5c84-45f9-95f2-b7fb545a3a5e None admin] RBAC: Authorizing `identity:create_grant(project_id=d9d68d310bf24449a5fe20d2dc264a58, user_id=8e86f38590824894a3b960e6774ff8a9, role_id=f42c599286be4e0fa632dc6769f95753)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:23.112529 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c1b453-5c84-45f9-95f2-b7fb545a3a5e None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:23.115666 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea126ec-db0d-4500-a0a1-6d94d4afed0e None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:23.126375 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 477/938] 10.222.0.26 () {58 vars in 1439 bytes} [Sat Aug 6 01:21:22 2022] PUT /identity/v3/projects/d9d68d310bf24449a5fe20d2dc264a58/users/8e86f38590824894a3b960e6774ff8a9/roles/f42c599286be4e0fa632dc6769f95753 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:21:23.137455 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c1b453-5c84-45f9-95f2-b7fb545a3a5e None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:23.173978 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb5c825-9fa8-47fa-8adb-b73f10e9dafb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:23.174828 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb5c825-9fa8-47fa-8adb-b73f10e9dafb None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:23.175110 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb5c825-9fa8-47fa-8adb-b73f10e9dafb None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:23.175752 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb5c825-9fa8-47fa-8adb-b73f10e9dafb None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:23.176804 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb5c825-9fa8-47fa-8adb-b73f10e9dafb None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:23.177137 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb5c825-9fa8-47fa-8adb-b73f10e9dafb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:23.179089 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb5c825-9fa8-47fa-8adb-b73f10e9dafb None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:23.189248 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 462/939] 10.222.0.26 () {56 vars in 1126 bytes} [Sat Aug 6 01:21:23 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:23.193380 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb5c825-9fa8-47fa-8adb-b73f10e9dafb None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:23.243788 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:23.281982 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:23.297425 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c1b453-5c84-45f9-95f2-b7fb545a3a5e None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:23.339864 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb5c825-9fa8-47fa-8adb-b73f10e9dafb None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:23.340077 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cda188-3ba7-4ec8-9d3a-9b2be834f712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:23.340720 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cda188-3ba7-4ec8-9d3a-9b2be834f712 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:23.341001 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cda188-3ba7-4ec8-9d3a-9b2be834f712 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:23.341313 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cda188-3ba7-4ec8-9d3a-9b2be834f712 None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:23.342003 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cda188-3ba7-4ec8-9d3a-9b2be834f712 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:23.342314 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cda188-3ba7-4ec8-9d3a-9b2be834f712 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:23.343689 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cda188-3ba7-4ec8-9d3a-9b2be834f712 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:23.353075 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 478/940] 10.222.0.26 () {56 vars in 1122 bytes} [Sat Aug 6 01:21:23 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:23.355413 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cda188-3ba7-4ec8-9d3a-9b2be834f712 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:23.383178 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b0d50a-9e37-4ebf-9985-255969d43c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:23.384080 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b0d50a-9e37-4ebf-9985-255969d43c6a None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:23.384536 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b0d50a-9e37-4ebf-9985-255969d43c6a None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:23.385544 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b0d50a-9e37-4ebf-9985-255969d43c6a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:23.386479 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b0d50a-9e37-4ebf-9985-255969d43c6a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:23.388042 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b0d50a-9e37-4ebf-9985-255969d43c6a None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:23.423088 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 463/941] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:21:23 2022] POST /identity/v3/endpoints => generated 347 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:21:23.475742 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cda188-3ba7-4ec8-9d3a-9b2be834f712 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:23.526822 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc8e127-898b-4b8b-8114-c72eeac35412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:23.527764 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc8e127-898b-4b8b-8114-c72eeac35412 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:23.528142 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc8e127-898b-4b8b-8114-c72eeac35412 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:23.528516 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc8e127-898b-4b8b-8114-c72eeac35412 None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:23.529366 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc8e127-898b-4b8b-8114-c72eeac35412 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:23.529775 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc8e127-898b-4b8b-8114-c72eeac35412 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:23.531322 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc8e127-898b-4b8b-8114-c72eeac35412 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:23.538815 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 479/942] 10.222.0.26 () {56 vars in 1122 bytes} [Sat Aug 6 01:21:23 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:21:23.545527 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b0d50a-9e37-4ebf-9985-255969d43c6a None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:23.644819 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:23.667008 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b0d50a-9e37-4ebf-9985-255969d43c6a None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:23.703530 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771d2dd9-cd6d-4bf6-942f-1a52cf174aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:23.704163 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771d2dd9-cd6d-4bf6-942f-1a52cf174aa4 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:23.704449 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771d2dd9-cd6d-4bf6-942f-1a52cf174aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:23.704729 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771d2dd9-cd6d-4bf6-942f-1a52cf174aa4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:23.705521 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771d2dd9-cd6d-4bf6-942f-1a52cf174aa4 None admin] RBAC: Adding query filter params (interface=public, service_id=c12c62708a51468e84396829b97276dd, region_id=RegionOne) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:23.705837 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771d2dd9-cd6d-4bf6-942f-1a52cf174aa4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:23.707200 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771d2dd9-cd6d-4bf6-942f-1a52cf174aa4 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:23.713322 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 464/943] 10.222.0.26 () {56 vars in 1251 bytes} [Sat Aug 6 01:21:23 2022] GET /identity/v3/endpoints?service_id=c12c62708a51468e84396829b97276dd&interface=public®ion_id=RegionOne => generated 194 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:23.718662 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc8e127-898b-4b8b-8114-c72eeac35412 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:23.743798 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:23.828895 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc8e127-898b-4b8b-8114-c72eeac35412 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:23.864845 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e577e71-b42c-4c5b-92cb-b9552be96b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:23.865545 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e577e71-b42c-4c5b-92cb-b9552be96b70 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:23.866095 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e577e71-b42c-4c5b-92cb-b9552be96b70 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:23.866445 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e577e71-b42c-4c5b-92cb-b9552be96b70 None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:23.867178 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e577e71-b42c-4c5b-92cb-b9552be96b70 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:23.868614 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e577e71-b42c-4c5b-92cb-b9552be96b70 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:23.875873 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 480/944] 10.222.0.26 () {56 vars in 1087 bytes} [Sat Aug 6 01:21:23 2022] GET /identity/v3/services => generated 1497 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:21:23.895887 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef59439-fbda-4367-bd79-e16979f761f4 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:23.896177 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef59439-fbda-4367-bd79-e16979f761f4 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:23.896460 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef59439-fbda-4367-bd79-e16979f761f4 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:23.898128 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 465/945] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:21:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:21:23.907047 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c657814-bb84-41a6-bf51-633d93696900 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:23.907387 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c657814-bb84-41a6-bf51-633d93696900 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:23.907636 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c657814-bb84-41a6-bf51-633d93696900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:23.928589 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-8c657814-bb84-41a6-bf51-633d93696900 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:21:23.929955 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-8c657814-bb84-41a6-bf51-633d93696900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:21:24.007775 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 481/946] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:21:23 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:21:24.031647 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef59439-fbda-4367-bd79-e16979f761f4 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:24.142929 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef59439-fbda-4367-bd79-e16979f761f4 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:24.178485 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a88467-7752-498a-86a8-0bfed9ec1c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:24.179118 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a88467-7752-498a-86a8-0bfed9ec1c97 None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:24.179406 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a88467-7752-498a-86a8-0bfed9ec1c97 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:24.179685 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a88467-7752-498a-86a8-0bfed9ec1c97 None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:24.180384 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a88467-7752-498a-86a8-0bfed9ec1c97 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:24.182460 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a88467-7752-498a-86a8-0bfed9ec1c97 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:24.185342 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: WARNING keystone.server.flask.common [None req-f6a88467-7752-498a-86a8-0bfed9ec1c97 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 06 01:21:24.205071 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.sql.core [None req-f6a88467-7752-498a-86a8-0bfed9ec1c97 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 06 01:21:24.205071 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 06 01:21:24.205071 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [parameters: {'id': 'd9291cb7a2e44760ac555218d82a2b50', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 06 01:21:24.205071 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80203) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 06 01:21:24.221947 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-f6a88467-7752-498a-86a8-0bfed9ec1c97 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id d9291cb7a2e44760ac555218d82a2b50.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id d9291cb7a2e44760ac555218d82a2b50. Aug 06 01:21:24.221947 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:24.221947 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 06 01:21:24.221947 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 06 01:21:24.221947 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 06 01:21:24.221947 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 06 01:21:24.221947 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 06 01:21:24.221947 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application result = self._query(query) Aug 06 01:21:24.221947 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 06 01:21:24.221947 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application conn.query(q) Aug 06 01:21:24.221947 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 06 01:21:24.221947 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 06 01:21:24.221947 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 06 01:21:24.222841 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application result.read() Aug 06 01:21:24.222841 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 06 01:21:24.222841 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 06 01:21:24.222841 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 06 01:21:24.222841 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 06 01:21:24.222841 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 06 01:21:24.222841 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 06 01:21:24.222841 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 06 01:21:24.222841 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 06 01:21:24.222841 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 06 01:21:24.222841 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:21:24.222841 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 06 01:21:24.222841 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:21:24.222841 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:24.222841 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 06 01:21:24.222841 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 06 01:21:24.222841 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 06 01:21:24.222841 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 06 01:21:24.223810 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 06 01:21:24.223810 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application next(self.gen) Aug 06 01:21:24.223810 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 06 01:21:24.223810 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application yield resource Aug 06 01:21:24.223810 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 06 01:21:24.223810 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application next(self.gen) Aug 06 01:21:24.223810 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 06 01:21:24.223810 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self.session.rollback() Aug 06 01:21:24.223810 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 06 01:21:24.223810 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self.force_reraise() Aug 06 01:21:24.223810 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 06 01:21:24.223810 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise self.value Aug 06 01:21:24.223810 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 06 01:21:24.223810 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 06 01:21:24.223810 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 06 01:21:24.223810 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application session.commit() Aug 06 01:21:24.223810 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 06 01:21:24.224703 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 06 01:21:24.224703 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 06 01:21:24.224703 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._prepare_impl() Aug 06 01:21:24.224703 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 06 01:21:24.224703 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self.session.flush() Aug 06 01:21:24.224703 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 06 01:21:24.224703 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._flush(objects) Aug 06 01:21:24.224703 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 06 01:21:24.224703 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 06 01:21:24.224703 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 06 01:21:24.224703 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application compat.raise_( Aug 06 01:21:24.224703 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 06 01:21:24.224703 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception Aug 06 01:21:24.224703 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 06 01:21:24.224703 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application flush_context.execute() Aug 06 01:21:24.224703 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 06 01:21:24.224703 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rec.execute(self) Aug 06 01:21:24.226916 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 06 01:21:24.226916 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 06 01:21:24.226916 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 06 01:21:24.226916 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 06 01:21:24.226916 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 06 01:21:24.226916 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 06 01:21:24.226916 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 06 01:21:24.226916 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 06 01:21:24.226916 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 06 01:21:24.226916 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 06 01:21:24.226916 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 06 01:21:24.226916 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 06 01:21:24.226916 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 06 01:21:24.226916 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 06 01:21:24.226916 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 06 01:21:24.229604 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 06 01:21:24.229604 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 06 01:21:24.229604 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception Aug 06 01:21:24.229604 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 06 01:21:24.229604 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 06 01:21:24.229604 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 06 01:21:24.229604 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 06 01:21:24.229604 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 06 01:21:24.229604 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application result = self._query(query) Aug 06 01:21:24.229604 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 06 01:21:24.229604 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application conn.query(q) Aug 06 01:21:24.229604 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 06 01:21:24.229604 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 06 01:21:24.229604 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 06 01:21:24.229604 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application result.read() Aug 06 01:21:24.229604 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 06 01:21:24.229604 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 06 01:21:24.231908 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 06 01:21:24.231908 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 06 01:21:24.231908 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 06 01:21:24.231908 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 06 01:21:24.231908 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 06 01:21:24.231908 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 06 01:21:24.231908 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 06 01:21:24.231908 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 06 01:21:24.231908 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [parameters: {'id': 'd9291cb7a2e44760ac555218d82a2b50', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 06 01:21:24.231908 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 06 01:21:24.231908 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:21:24.231908 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 01:21:24.231908 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:21:24.231908 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:24.231908 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 06 01:21:24.239459 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 06 01:21:24.239459 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 06 01:21:24.239459 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 06 01:21:24.239459 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 06 01:21:24.239459 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:21:24.239459 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 01:21:24.239459 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:21:24.239459 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:24.239459 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:21:24.239459 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:21:24.239459 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:21:24.239459 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:21:24.239459 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:21:24.239459 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:21:24.239459 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:21:24.239459 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:21:24.239459 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:21:24.241772 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:21:24.241772 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 06 01:21:24.241772 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 06 01:21:24.241772 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:21:24.241772 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:21:24.241772 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 06 01:21:24.241772 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 06 01:21:24.241772 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id d9291cb7a2e44760ac555218d82a2b50. Aug 06 01:21:24.241772 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:21:24.241772 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 466/947] 10.222.0.26 () {60 vars in 1140 bytes} [Sat Aug 6 01:21:24 2022] POST /identity/v3/projects => generated 262 bytes in 192 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:21:24.245408 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c657814-bb84-41a6-bf51-633d93696900 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:24.245408 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:24.337545 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c657814-bb84-41a6-bf51-633d93696900 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:24.375790 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42793e31-7854-4536-b439-32234a8165b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:24.376560 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42793e31-7854-4536-b439-32234a8165b8 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:24.376900 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42793e31-7854-4536-b439-32234a8165b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:24.377214 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42793e31-7854-4536-b439-32234a8165b8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:24.384687 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42793e31-7854-4536-b439-32234a8165b8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:24.386329 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42793e31-7854-4536-b439-32234a8165b8 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:24.399416 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-42793e31-7854-4536-b439-32234a8165b8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:21:24.399416 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:24.399416 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:21:24.399416 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:21:24.399416 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:21:24.399416 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:21:24.399416 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:21:24.399416 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:21:24.399416 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:21:24.399416 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:21:24.399416 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:21:24.399416 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:21:24.399416 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 01:21:24.399416 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 01:21:24.399416 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 01:21:24.400320 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 01:21:24.400320 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:21:24.400320 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:21:24.400320 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:21:24.400320 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:21:24.400320 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:21:24.400320 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:21:24.400320 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:21:24.400320 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:21:24.400320 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:21:24.400320 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:21:24.400320 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:21:24.400320 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:21:24.400320 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:21:24.400320 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:21:24.400320 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:21:24.400320 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:21:24.403102 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 01:21:24.403102 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 01:21:24.403102 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 01:21:24.403102 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 01:21:24.403102 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 01:21:24.403102 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 01:21:24.403102 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:21:24.403102 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:21:24.403102 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 482/948] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Aug 6 01:21:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:21:24.407685 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a88467-7752-498a-86a8-0bfed9ec1c97 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:24.429560 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:24.517396 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a88467-7752-498a-86a8-0bfed9ec1c97 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:24.552196 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4378085-2b73-456e-ae19-e72e7954f666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:24.552889 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4378085-2b73-456e-ae19-e72e7954f666 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:24.553205 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4378085-2b73-456e-ae19-e72e7954f666 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:24.553537 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4378085-2b73-456e-ae19-e72e7954f666 None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:24.554281 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4378085-2b73-456e-ae19-e72e7954f666 None admin] RBAC: Adding query filter params (name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:24.554593 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4378085-2b73-456e-ae19-e72e7954f666 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:24.556228 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4378085-2b73-456e-ae19-e72e7954f666 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:24.578476 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 467/949] 10.222.0.26 () {56 vars in 1112 bytes} [Sat Aug 6 01:21:24 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:25.319997 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902b42d2-1c7f-4c42-839d-8dc7253a6575 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:25.321039 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902b42d2-1c7f-4c42-839d-8dc7253a6575 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:25.321582 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902b42d2-1c7f-4c42-839d-8dc7253a6575 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:25.323539 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 483/950] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:21:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:21:25.328939 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ef2e7f-88e1-419b-a277-2c502af12084 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:25.329805 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ef2e7f-88e1-419b-a277-2c502af12084 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:25.330291 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ef2e7f-88e1-419b-a277-2c502af12084 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:25.350048 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-01ef2e7f-88e1-419b-a277-2c502af12084 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:21:25.352140 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-01ef2e7f-88e1-419b-a277-2c502af12084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:21:25.419405 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:25.426444 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 468/951] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:21:25 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:21:25.450798 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902b42d2-1c7f-4c42-839d-8dc7253a6575 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:25.525189 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:25.564088 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902b42d2-1c7f-4c42-839d-8dc7253a6575 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:25.565394 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a99425b-41e0-48fe-b866-0eba9f6ebb33 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:25.565693 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a99425b-41e0-48fe-b866-0eba9f6ebb33 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:25.565973 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a99425b-41e0-48fe-b866-0eba9f6ebb33 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:25.567630 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 469/952] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:21:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 01:21:25.573530 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbb9e10-bd56-43aa-92fa-296f8c9c85a0 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:25.573946 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbb9e10-bd56-43aa-92fa-296f8c9c85a0 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:25.574723 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbb9e10-bd56-43aa-92fa-296f8c9c85a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:25.601523 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-8dbb9e10-bd56-43aa-92fa-296f8c9c85a0 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:21:25.603199 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-8dbb9e10-bd56-43aa-92fa-296f8c9c85a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:21:25.604258 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f0a059-495f-4e70-9ff9-403f3ab119c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:25.604908 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f0a059-495f-4e70-9ff9-403f3ab119c6 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:25.605213 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f0a059-495f-4e70-9ff9-403f3ab119c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:25.605488 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f0a059-495f-4e70-9ff9-403f3ab119c6 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:25.617650 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f0a059-495f-4e70-9ff9-403f3ab119c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:21:25.623821 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-57f0a059-495f-4e70-9ff9-403f3ab119c6 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 06 01:21:25.623821 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:25.623821 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:21:25.623821 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:21:25.623821 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:21:25.623821 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:21:25.623821 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:21:25.623821 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:21:25.623821 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:21:25.623821 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:21:25.623821 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:21:25.623821 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:21:25.623821 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:21:25.623821 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:21:25.623821 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:21:25.623821 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:21:25.624792 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:21:25.624792 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:21:25.624792 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:21:25.624792 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:21:25.624792 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:21:25.624792 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:21:25.624792 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:21:25.624792 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:21:25.624792 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:21:25.624792 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:21:25.624792 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:21:25.624792 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:21:25.624792 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:21:25.624792 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:21:25.624792 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:21:25.624792 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:21:25.625693 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:21:25.625693 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:21:25.625693 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:21:25.625693 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:21:25.625693 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:21:25.625693 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:21:25.625693 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:21:25.625693 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:21:25.625693 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:21:25.625693 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:21:25.625693 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 06 01:21:25.625693 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:21:25.625693 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 484/953] 10.222.0.26 () {56 vars in 1120 bytes} [Sat Aug 6 01:21:25 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:21:25.631352 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f0a059-495f-4e70-9ff9-403f3ab119c6 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:25.685723 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 470/954] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:21:25 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 06 01:21:25.713262 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbb9e10-bd56-43aa-92fa-296f8c9c85a0 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:25.752828 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f0a059-495f-4e70-9ff9-403f3ab119c6 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:25.813077 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:25.825204 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762fd908-2471-4d63-ad36-b7bbc0d399e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:25.825838 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762fd908-2471-4d63-ad36-b7bbc0d399e9 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:25.829070 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762fd908-2471-4d63-ad36-b7bbc0d399e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:25.829070 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762fd908-2471-4d63-ad36-b7bbc0d399e9 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:25.829070 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762fd908-2471-4d63-ad36-b7bbc0d399e9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:25.829070 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762fd908-2471-4d63-ad36-b7bbc0d399e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:25.829070 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762fd908-2471-4d63-ad36-b7bbc0d399e9 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:25.849139 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 485/955] 10.222.0.26 () {56 vars in 1115 bytes} [Sat Aug 6 01:21:25 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:21:25.850183 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762fd908-2471-4d63-ad36-b7bbc0d399e9 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:25.893076 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbb9e10-bd56-43aa-92fa-296f8c9c85a0 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:25.937453 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:25.950542 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0a561b-26b5-4768-9ae9-981961dfe72d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:25.951192 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0a561b-26b5-4768-9ae9-981961dfe72d None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:25.951472 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0a561b-26b5-4768-9ae9-981961dfe72d None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:25.951747 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0a561b-26b5-4768-9ae9-981961dfe72d None admin] PATH_INFO: `/v3/services/compute` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:25.952373 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0a561b-26b5-4768-9ae9-981961dfe72d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:25.954296 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0a561b-26b5-4768-9ae9-981961dfe72d None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:25.971282 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-2b0a561b-26b5-4768-9ae9-981961dfe72d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 06 01:21:25.971282 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:25.971282 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 01:21:25.971282 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 01:21:25.971282 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 01:21:25.971282 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 01:21:25.971282 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 01:21:25.971282 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 01:21:25.971282 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 06 01:21:25.971282 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:21:25.971282 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 01:21:25.971282 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:21:25.971282 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:25.971282 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:21:25.971282 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:21:25.971282 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:21:25.971282 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:21:25.972324 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:21:25.972324 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:21:25.972324 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:21:25.972324 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:21:25.972324 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:21:25.972324 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:21:25.972324 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 01:21:25.972324 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 01:21:25.972324 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 01:21:25.972324 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 01:21:25.972324 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:21:25.972324 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:21:25.972324 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:21:25.972324 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:21:25.972324 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:21:25.972324 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:21:25.979842 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:21:25.979842 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:21:25.979842 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:21:25.979842 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:21:25.979842 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:21:25.979842 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:21:25.979842 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:21:25.979842 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:21:25.979842 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:21:25.979842 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:21:25.979842 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 01:21:25.979842 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 01:21:25.979842 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 06 01:21:25.979842 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:21:25.979842 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 471/956] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Aug 6 01:21:25 2022] GET /identity/v3/services/compute => generated 88 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:21:25.987237 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0a561b-26b5-4768-9ae9-981961dfe72d None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:26.031073 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762fd908-2471-4d63-ad36-b7bbc0d399e9 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:26.074668 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1268fc6c-2d63-4676-a45e-a6a0253a05a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:26.075415 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1268fc6c-2d63-4676-a45e-a6a0253a05a7 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:26.075643 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1268fc6c-2d63-4676-a45e-a6a0253a05a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:26.075929 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1268fc6c-2d63-4676-a45e-a6a0253a05a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:26.089357 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1268fc6c-2d63-4676-a45e-a6a0253a05a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:26.090883 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1268fc6c-2d63-4676-a45e-a6a0253a05a7 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:26.107145 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-1268fc6c-2d63-4676-a45e-a6a0253a05a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:21:26.107145 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:26.107145 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:21:26.107145 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:21:26.107145 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:21:26.107145 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:21:26.107145 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:21:26.107145 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:21:26.107145 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:21:26.107145 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:21:26.107145 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:21:26.107145 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:21:26.107145 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:21:26.107145 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:21:26.107145 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:21:26.108152 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:21:26.108152 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:21:26.108152 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:21:26.108152 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:21:26.108152 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:21:26.108152 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:21:26.108152 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:21:26.108152 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:21:26.108152 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:21:26.108152 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:21:26.108152 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:21:26.108152 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:21:26.108152 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:21:26.108152 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:21:26.108152 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:21:26.108152 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:21:26.108152 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:21:26.109197 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:21:26.109197 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:21:26.109197 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:21:26.109197 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:21:26.109197 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 486/957] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:21:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:21:26.110165 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabc7c08-8f01-4fb6-871e-cdaf1f265c5b None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:26.110165 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabc7c08-8f01-4fb6-871e-cdaf1f265c5b None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:26.110165 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabc7c08-8f01-4fb6-871e-cdaf1f265c5b None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:26.115699 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 487/958] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:21:26 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:21:26.115983 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabc7c08-8f01-4fb6-871e-cdaf1f265c5b None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:26.121703 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0a561b-26b5-4768-9ae9-981961dfe72d None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:26.164544 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904c8584-3438-4a42-bf88-d82ef5690cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:26.165206 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904c8584-3438-4a42-bf88-d82ef5690cf7 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:26.165496 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904c8584-3438-4a42-bf88-d82ef5690cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:26.165835 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904c8584-3438-4a42-bf88-d82ef5690cf7 None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:26.166833 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904c8584-3438-4a42-bf88-d82ef5690cf7 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:26.167150 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904c8584-3438-4a42-bf88-d82ef5690cf7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:26.168526 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904c8584-3438-4a42-bf88-d82ef5690cf7 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:26.177182 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:26.183811 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 472/959] 10.222.0.26 () {56 vars in 1112 bytes} [Sat Aug 6 01:21:25 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:26.183811 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411d7fb7-cc8b-49e8-a821-f27c58914093 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:26.183811 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411d7fb7-cc8b-49e8-a821-f27c58914093 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:26.183811 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411d7fb7-cc8b-49e8-a821-f27c58914093 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:26.200311 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-411d7fb7-cc8b-49e8-a821-f27c58914093 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:21:26.202064 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-411d7fb7-cc8b-49e8-a821-f27c58914093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:21:26.252324 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabc7c08-8f01-4fb6-871e-cdaf1f265c5b None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:26.273517 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:26.293231 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 473/960] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:21:26 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:21:26.294356 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411d7fb7-cc8b-49e8-a821-f27c58914093 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:26.310199 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e3c987-d982-43f5-b7d7-a539520f88fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:26.311505 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e3c987-d982-43f5-b7d7-a539520f88fa None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:26.312226 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e3c987-d982-43f5-b7d7-a539520f88fa None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:26.312675 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e3c987-d982-43f5-b7d7-a539520f88fa None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:26.313817 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e3c987-d982-43f5-b7d7-a539520f88fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:26.314707 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e3c987-d982-43f5-b7d7-a539520f88fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:26.316242 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e3c987-d982-43f5-b7d7-a539520f88fa None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:26.333688 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 488/961] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Aug 6 01:21:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:26.335189 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e3c987-d982-43f5-b7d7-a539520f88fa None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:26.408587 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411d7fb7-cc8b-49e8-a821-f27c58914093 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:26.446003 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882339c2-fa30-482f-8966-cfed1d337a76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:26.446633 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882339c2-fa30-482f-8966-cfed1d337a76 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:26.446921 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882339c2-fa30-482f-8966-cfed1d337a76 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:26.447202 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882339c2-fa30-482f-8966-cfed1d337a76 None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:26.448501 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882339c2-fa30-482f-8966-cfed1d337a76 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:26.448818 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882339c2-fa30-482f-8966-cfed1d337a76 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:26.450241 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882339c2-fa30-482f-8966-cfed1d337a76 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:26.454982 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 474/962] 10.222.0.26 () {56 vars in 1112 bytes} [Sat Aug 6 01:21:26 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:26.455756 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882339c2-fa30-482f-8966-cfed1d337a76 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:26.456811 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e3c987-d982-43f5-b7d7-a539520f88fa None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:26.468081 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:26.506306 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b4f634-c3d3-496f-b53d-9a09d7e6a841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:26.507028 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b4f634-c3d3-496f-b53d-9a09d7e6a841 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:26.507321 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b4f634-c3d3-496f-b53d-9a09d7e6a841 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:26.507701 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b4f634-c3d3-496f-b53d-9a09d7e6a841 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:26.508572 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b4f634-c3d3-496f-b53d-9a09d7e6a841 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:26.510126 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b4f634-c3d3-496f-b53d-9a09d7e6a841 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:26.515902 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:26.523386 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-e8b4f634-c3d3-496f-b53d-9a09d7e6a841 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 06 01:21:26.523386 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:26.523386 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 01:21:26.523386 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 01:21:26.523386 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 01:21:26.523386 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 01:21:26.523386 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 01:21:26.523386 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 01:21:26.523386 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 06 01:21:26.523386 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:21:26.523386 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 01:21:26.523386 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:21:26.523386 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:26.523386 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:21:26.523386 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:21:26.523386 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:21:26.523386 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:21:26.524573 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:21:26.524573 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:21:26.524573 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:21:26.524573 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:21:26.524573 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:21:26.524573 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:21:26.524573 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 01:21:26.524573 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 01:21:26.524573 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 01:21:26.524573 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 01:21:26.524573 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:21:26.524573 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:21:26.524573 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:21:26.524573 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:21:26.524573 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:21:26.524573 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:21:26.526014 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:21:26.526014 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:21:26.526014 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:21:26.526014 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:21:26.526014 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:21:26.526014 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:21:26.526014 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:21:26.526014 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:21:26.526014 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:21:26.526014 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:21:26.526014 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 01:21:26.526014 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 01:21:26.526014 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 06 01:21:26.526014 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:21:26.529237 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 489/963] 10.222.0.26 () {56 vars in 1126 bytes} [Sat Aug 6 01:21:26 2022] GET /identity/v3/services/object-store => generated 93 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:21:26.535183 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b4f634-c3d3-496f-b53d-9a09d7e6a841 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:26.578047 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882339c2-fa30-482f-8966-cfed1d337a76 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:26.618528 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c08eff-f56f-4967-8888-c66b25e266c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:26.619545 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c08eff-f56f-4967-8888-c66b25e266c2 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:26.620026 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c08eff-f56f-4967-8888-c66b25e266c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:26.620493 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c08eff-f56f-4967-8888-c66b25e266c2 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:26.629392 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c08eff-f56f-4967-8888-c66b25e266c2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:26.631154 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c08eff-f56f-4967-8888-c66b25e266c2 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:26.645079 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-51c08eff-f56f-4967-8888-c66b25e266c2 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 06 01:21:26.645079 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:26.645079 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:21:26.645079 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:21:26.645079 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:21:26.645079 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:21:26.645079 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:21:26.645079 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:21:26.645079 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:21:26.645079 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:21:26.645079 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:21:26.645079 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:21:26.645079 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 01:21:26.645079 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 01:21:26.645079 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 01:21:26.645079 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 01:21:26.646335 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:21:26.646335 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:21:26.646335 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 01:21:26.646335 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:21:26.646335 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 01:21:26.646335 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:21:26.646335 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:21:26.646335 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:21:26.646335 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:21:26.646335 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:21:26.646335 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:21:26.646335 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:21:26.646335 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:21:26.646335 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:21:26.646335 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:21:26.646335 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:21:26.646335 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:21:26.646335 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:21:26.647514 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:21:26.647514 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:21:26.647514 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 01:21:26.647514 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 01:21:26.647514 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 01:21:26.647514 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 01:21:26.647514 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 01:21:26.647514 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 01:21:26.647514 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 06 01:21:26.647514 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:21:26.649841 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 475/964] 10.222.0.26 () {56 vars in 1117 bytes} [Sat Aug 6 01:21:26 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:21:26.656353 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c08eff-f56f-4967-8888-c66b25e266c2 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:26.668326 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b4f634-c3d3-496f-b53d-9a09d7e6a841 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:26.705438 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38263cf2-a0ab-4256-bd0d-d88d2e68548d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:26.706062 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38263cf2-a0ab-4256-bd0d-d88d2e68548d None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:26.706347 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38263cf2-a0ab-4256-bd0d-d88d2e68548d None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:26.706657 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38263cf2-a0ab-4256-bd0d-d88d2e68548d None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:26.707311 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38263cf2-a0ab-4256-bd0d-d88d2e68548d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:26.707647 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38263cf2-a0ab-4256-bd0d-d88d2e68548d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:26.709019 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38263cf2-a0ab-4256-bd0d-d88d2e68548d None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:26.715392 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 490/965] 10.222.0.26 () {56 vars in 1122 bytes} [Sat Aug 6 01:21:26 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:26.717306 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaa5e91-0ee8-4e11-acf5-fe8e9a275005 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:26.718008 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaa5e91-0ee8-4e11-acf5-fe8e9a275005 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:26.718446 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaa5e91-0ee8-4e11-acf5-fe8e9a275005 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:26.720385 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 491/966] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:21:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:21:26.721286 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaa5e91-0ee8-4e11-acf5-fe8e9a275005 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:26.758927 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:26.777922 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c08eff-f56f-4967-8888-c66b25e266c2 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:26.789428 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:26.817754 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a67c39f-848e-4723-a46d-785be9528c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:26.818768 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a67c39f-848e-4723-a46d-785be9528c61 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:26.819124 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a67c39f-848e-4723-a46d-785be9528c61 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:26.819474 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a67c39f-848e-4723-a46d-785be9528c61 None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:26.820340 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a67c39f-848e-4723-a46d-785be9528c61 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:26.820722 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a67c39f-848e-4723-a46d-785be9528c61 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:26.823583 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a67c39f-848e-4723-a46d-785be9528c61 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:26.834819 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaa5e91-0ee8-4e11-acf5-fe8e9a275005 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:26.848034 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 476/967] 10.222.0.26 () {56 vars in 1149 bytes} [Sat Aug 6 01:21:26 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:21:26.849381 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55236ac-b6ca-4de7-b917-ce67fd397dad None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:26.849699 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55236ac-b6ca-4de7-b917-ce67fd397dad None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:26.849983 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55236ac-b6ca-4de7-b917-ce67fd397dad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:26.868440 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-e55236ac-b6ca-4de7-b917-ce67fd397dad None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:21:26.870290 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-e55236ac-b6ca-4de7-b917-ce67fd397dad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:21:26.879691 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243404c7-25cf-4515-926f-e87a95c1f4e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:26.880309 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243404c7-25cf-4515-926f-e87a95c1f4e5 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:26.880595 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243404c7-25cf-4515-926f-e87a95c1f4e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:26.880876 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243404c7-25cf-4515-926f-e87a95c1f4e5 None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:26.881660 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243404c7-25cf-4515-926f-e87a95c1f4e5 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:26.881908 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243404c7-25cf-4515-926f-e87a95c1f4e5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:26.883265 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243404c7-25cf-4515-926f-e87a95c1f4e5 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:26.888779 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 492/968] 10.222.0.26 () {56 vars in 1122 bytes} [Sat Aug 6 01:21:26 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:26.889888 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243404c7-25cf-4515-926f-e87a95c1f4e5 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:26.944093 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 477/969] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:21:26 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:21:26.945061 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55236ac-b6ca-4de7-b917-ce67fd397dad None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:27.007609 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:27.014424 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243404c7-25cf-4515-926f-e87a95c1f4e5 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:27.055491 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8986a58-7559-4275-95d6-23c64e4b3228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:27.056364 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8986a58-7559-4275-95d6-23c64e4b3228 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:27.057281 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8986a58-7559-4275-95d6-23c64e4b3228 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:27.057579 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8986a58-7559-4275-95d6-23c64e4b3228 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:27.060479 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:27.064752 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55236ac-b6ca-4de7-b917-ce67fd397dad None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:27.102085 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd888f2e-e53f-48a2-a7b7-3f0b2717c2ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:27.102801 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd888f2e-e53f-48a2-a7b7-3f0b2717c2ff None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:27.103205 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd888f2e-e53f-48a2-a7b7-3f0b2717c2ff None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:27.104087 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd888f2e-e53f-48a2-a7b7-3f0b2717c2ff None admin] PATH_INFO: `/v3/endpoints` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:27.104908 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd888f2e-e53f-48a2-a7b7-3f0b2717c2ff None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:27.106376 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd888f2e-e53f-48a2-a7b7-3f0b2717c2ff None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:27.133189 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 478/970] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:21:26 2022] POST /identity/v3/endpoints => generated 349 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:21:27.133189 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd888f2e-e53f-48a2-a7b7-3f0b2717c2ff None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:27.141830 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8986a58-7559-4275-95d6-23c64e4b3228 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:27.143571 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8986a58-7559-4275-95d6-23c64e4b3228 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:27.253172 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:27.261209 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 493/971] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Aug 6 01:21:26 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 06 01:21:27.267112 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8986a58-7559-4275-95d6-23c64e4b3228 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:27.274855 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd888f2e-e53f-48a2-a7b7-3f0b2717c2ff None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:27.326365 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ebb8e4-6782-4da5-860e-b9600e457e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:27.327571 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ebb8e4-6782-4da5-860e-b9600e457e11 None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:27.328053 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ebb8e4-6782-4da5-860e-b9600e457e11 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:27.328502 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ebb8e4-6782-4da5-860e-b9600e457e11 None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:27.329576 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ebb8e4-6782-4da5-860e-b9600e457e11 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:27.332315 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ebb8e4-6782-4da5-860e-b9600e457e11 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:27.349403 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.sql.core [None req-57ebb8e4-6782-4da5-860e-b9600e457e11 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 06 01:21:27.349403 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 06 01:21:27.349403 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [parameters: {'user_id': 'ff5b17e3afa94a1d93cb86af6976d8e2', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 06 01:21:27.349403 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80203) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 06 01:21:27.357760 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-57ebb8e4-6782-4da5-860e-b9600e457e11 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 06 01:21:27.357760 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:27.357760 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 06 01:21:27.357760 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 06 01:21:27.357760 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 06 01:21:27.357760 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 06 01:21:27.357760 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 06 01:21:27.357760 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application result = self._query(query) Aug 06 01:21:27.357760 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 06 01:21:27.357760 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application conn.query(q) Aug 06 01:21:27.357760 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 06 01:21:27.357760 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 06 01:21:27.357760 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 06 01:21:27.357760 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application result.read() Aug 06 01:21:27.357760 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 06 01:21:27.358623 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 06 01:21:27.358623 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 06 01:21:27.358623 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 06 01:21:27.358623 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 06 01:21:27.358623 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 06 01:21:27.358623 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 06 01:21:27.358623 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 06 01:21:27.358623 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 06 01:21:27.358623 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:21:27.358623 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 06 01:21:27.358623 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:21:27.358623 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:27.358623 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 06 01:21:27.358623 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 06 01:21:27.358623 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 06 01:21:27.358623 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 06 01:21:27.358623 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 06 01:21:27.358623 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application next(self.gen) Aug 06 01:21:27.359607 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 06 01:21:27.359607 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application yield resource Aug 06 01:21:27.359607 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 06 01:21:27.359607 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application next(self.gen) Aug 06 01:21:27.359607 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 06 01:21:27.359607 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self.session.rollback() Aug 06 01:21:27.359607 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 06 01:21:27.359607 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self.force_reraise() Aug 06 01:21:27.359607 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 06 01:21:27.359607 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise self.value Aug 06 01:21:27.359607 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 06 01:21:27.359607 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 06 01:21:27.359607 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 06 01:21:27.359607 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application session.commit() Aug 06 01:21:27.359607 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 06 01:21:27.359607 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 06 01:21:27.359607 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 06 01:21:27.360515 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._prepare_impl() Aug 06 01:21:27.360515 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 06 01:21:27.360515 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self.session.flush() Aug 06 01:21:27.360515 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 06 01:21:27.360515 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._flush(objects) Aug 06 01:21:27.360515 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 06 01:21:27.360515 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 06 01:21:27.360515 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 06 01:21:27.360515 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application compat.raise_( Aug 06 01:21:27.360515 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 06 01:21:27.360515 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception Aug 06 01:21:27.360515 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 06 01:21:27.360515 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application flush_context.execute() Aug 06 01:21:27.360515 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 06 01:21:27.360515 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rec.execute(self) Aug 06 01:21:27.360515 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 06 01:21:27.360515 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 06 01:21:27.362165 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 06 01:21:27.362165 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 06 01:21:27.362165 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 06 01:21:27.362165 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 06 01:21:27.362165 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 06 01:21:27.362165 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 06 01:21:27.362165 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 06 01:21:27.362165 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 06 01:21:27.362165 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 06 01:21:27.362165 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 06 01:21:27.362165 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 06 01:21:27.362165 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 06 01:21:27.362165 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 06 01:21:27.362165 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 06 01:21:27.362165 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 06 01:21:27.362165 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception Aug 06 01:21:27.363054 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 06 01:21:27.363054 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 06 01:21:27.363054 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 06 01:21:27.363054 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 06 01:21:27.363054 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 06 01:21:27.363054 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application result = self._query(query) Aug 06 01:21:27.363054 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 06 01:21:27.363054 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application conn.query(q) Aug 06 01:21:27.363054 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 06 01:21:27.363054 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 06 01:21:27.363054 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 06 01:21:27.363054 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application result.read() Aug 06 01:21:27.363054 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 06 01:21:27.363054 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 06 01:21:27.363054 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 06 01:21:27.363054 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 06 01:21:27.363902 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 06 01:21:27.363902 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 06 01:21:27.363902 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 06 01:21:27.363902 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 06 01:21:27.363902 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 06 01:21:27.363902 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 06 01:21:27.363902 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [parameters: {'user_id': 'ff5b17e3afa94a1d93cb86af6976d8e2', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 06 01:21:27.363902 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 06 01:21:27.363902 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:21:27.363902 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 01:21:27.363902 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:21:27.363902 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:27.363902 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:21:27.363902 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:21:27.363902 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:21:27.364737 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:21:27.364737 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:21:27.364737 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:21:27.364737 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:21:27.364737 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:21:27.364737 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:21:27.364737 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:21:27.364737 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 06 01:21:27.364737 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 06 01:21:27.364737 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:21:27.364737 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:21:27.364737 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 01:21:27.364737 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:21:27.364737 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 01:21:27.364737 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:21:27.364737 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 06 01:21:27.364737 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 06 01:21:27.365738 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 06 01:21:27.365738 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 06 01:21:27.365738 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 06 01:21:27.365738 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 06 01:21:27.365738 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 06 01:21:27.365738 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:21:27.365738 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 479/972] 10.222.0.26 () {60 vars in 1131 bytes} [Sat Aug 6 01:21:27 2022] POST /identity/v3/users => generated 160 bytes in 231 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:21:27.365738 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ebb8e4-6782-4da5-860e-b9600e457e11 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:27.404737 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8986a58-7559-4275-95d6-23c64e4b3228 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:27.437542 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:27.450505 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185327ca-2a41-4689-a615-4ac99b95e809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:27.451181 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185327ca-2a41-4689-a615-4ac99b95e809 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:27.451467 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185327ca-2a41-4689-a615-4ac99b95e809 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:27.451742 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185327ca-2a41-4689-a615-4ac99b95e809 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:27.465041 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185327ca-2a41-4689-a615-4ac99b95e809 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:27.466932 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185327ca-2a41-4689-a615-4ac99b95e809 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:27.483625 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-185327ca-2a41-4689-a615-4ac99b95e809 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:21:27.483625 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:27.483625 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:21:27.483625 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:21:27.483625 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:21:27.483625 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:21:27.483625 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:21:27.483625 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:21:27.483625 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:21:27.483625 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:21:27.483625 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:21:27.483625 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:21:27.483625 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:21:27.483625 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:21:27.483625 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:21:27.484548 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:21:27.484548 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:21:27.484548 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:21:27.484548 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:21:27.484548 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:21:27.484548 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:21:27.484548 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:21:27.484548 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:21:27.484548 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:21:27.484548 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:21:27.484548 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:21:27.484548 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:21:27.484548 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:21:27.484548 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:21:27.484548 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:21:27.484548 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:21:27.484548 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:21:27.485492 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:21:27.485492 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:21:27.485492 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:21:27.485492 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:21:27.487675 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 494/973] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:21:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:21:27.494536 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185327ca-2a41-4689-a615-4ac99b95e809 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:27.501688 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ebb8e4-6782-4da5-860e-b9600e457e11 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:27.548281 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011e4361-e0a5-46a5-85e8-78198aa5919b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:27.549262 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011e4361-e0a5-46a5-85e8-78198aa5919b None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:27.549551 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011e4361-e0a5-46a5-85e8-78198aa5919b None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:27.549826 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011e4361-e0a5-46a5-85e8-78198aa5919b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:27.553709 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:27.563536 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011e4361-e0a5-46a5-85e8-78198aa5919b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:27.566816 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011e4361-e0a5-46a5-85e8-78198aa5919b None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:27.580501 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-011e4361-e0a5-46a5-85e8-78198aa5919b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 06 01:21:27.580501 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:27.580501 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:21:27.580501 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:21:27.580501 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:21:27.580501 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:21:27.580501 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:21:27.580501 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:21:27.580501 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:21:27.580501 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:21:27.580501 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:21:27.580501 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:21:27.580501 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 01:21:27.580501 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 01:21:27.580501 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 01:21:27.580501 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 01:21:27.581704 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:21:27.581704 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:21:27.581704 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 01:21:27.581704 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:21:27.581704 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 01:21:27.581704 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:21:27.581704 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:21:27.581704 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:21:27.581704 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:21:27.581704 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:21:27.581704 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:21:27.581704 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:21:27.581704 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:21:27.581704 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:21:27.581704 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:21:27.581704 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:21:27.581704 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:21:27.581704 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:21:27.582674 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:21:27.582674 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:21:27.582674 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 01:21:27.582674 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 01:21:27.582674 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 01:21:27.582674 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 01:21:27.582674 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 01:21:27.582674 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 01:21:27.582674 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 06 01:21:27.582674 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:21:27.582674 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 480/974] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Aug 6 01:21:27 2022] GET /identity/v3/users/cinder => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:21:27.588701 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011e4361-e0a5-46a5-85e8-78198aa5919b None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:27.631462 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185327ca-2a41-4689-a615-4ac99b95e809 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:27.667808 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:27.682449 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd6f48c-492c-4085-a8a1-f53a5a43b8f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:27.683473 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd6f48c-492c-4085-a8a1-f53a5a43b8f1 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:27.684050 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd6f48c-492c-4085-a8a1-f53a5a43b8f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:27.684481 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd6f48c-492c-4085-a8a1-f53a5a43b8f1 None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:27.685631 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd6f48c-492c-4085-a8a1-f53a5a43b8f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:27.685946 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd6f48c-492c-4085-a8a1-f53a5a43b8f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:27.687324 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd6f48c-492c-4085-a8a1-f53a5a43b8f1 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:27.705880 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011e4361-e0a5-46a5-85e8-78198aa5919b None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:27.707733 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 495/975] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Aug 6 01:21:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:27.712695 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd6f48c-492c-4085-a8a1-f53a5a43b8f1 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:27.748956 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b99b742-0530-470b-a861-a4afb0239643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:27.749841 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b99b742-0530-470b-a861-a4afb0239643 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:27.750292 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b99b742-0530-470b-a861-a4afb0239643 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:27.750807 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b99b742-0530-470b-a861-a4afb0239643 None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:27.751818 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b99b742-0530-470b-a861-a4afb0239643 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:27.752273 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b99b742-0530-470b-a861-a4afb0239643 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:27.753862 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b99b742-0530-470b-a861-a4afb0239643 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:27.801132 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 481/976] 10.222.0.26 () {56 vars in 1101 bytes} [Sat Aug 6 01:21:27 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:27.826093 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd6f48c-492c-4085-a8a1-f53a5a43b8f1 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:27.837275 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:27.868102 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d3112e-4876-4e79-a286-ffa56ce3f952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:27.868766 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d3112e-4876-4e79-a286-ffa56ce3f952 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:27.869068 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d3112e-4876-4e79-a286-ffa56ce3f952 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:27.869426 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d3112e-4876-4e79-a286-ffa56ce3f952 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:27.877233 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d3112e-4876-4e79-a286-ffa56ce3f952 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:27.878758 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d3112e-4876-4e79-a286-ffa56ce3f952 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:27.892223 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-f7d3112e-4876-4e79-a286-ffa56ce3f952 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:21:27.892223 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:27.892223 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:21:27.892223 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:21:27.892223 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:21:27.892223 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:21:27.892223 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:21:27.892223 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:21:27.892223 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:21:27.892223 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:21:27.892223 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:21:27.892223 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:21:27.892223 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 01:21:27.892223 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 01:21:27.892223 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 01:21:27.893368 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 01:21:27.893368 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:21:27.893368 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:21:27.893368 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:21:27.893368 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:21:27.893368 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:21:27.893368 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:21:27.893368 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:21:27.893368 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:21:27.893368 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:21:27.893368 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:21:27.893368 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:21:27.893368 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:21:27.893368 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:21:27.893368 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:21:27.893368 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:21:27.893368 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:21:27.894533 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 01:21:27.894533 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 01:21:27.894533 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 01:21:27.894533 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 01:21:27.894533 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 01:21:27.894533 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 01:21:27.894533 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:21:27.894533 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:21:27.894533 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 496/977] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Aug 6 01:21:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:21:27.899719 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b99b742-0530-470b-a861-a4afb0239643 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:28.009111 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b99b742-0530-470b-a861-a4afb0239643 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:28.038993 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:28.046925 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02eb31f7-46c1-43b6-bcac-fbb813663de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:28.047638 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eb31f7-46c1-43b6-bcac-fbb813663de9 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:28.048032 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eb31f7-46c1-43b6-bcac-fbb813663de9 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:28.048344 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eb31f7-46c1-43b6-bcac-fbb813663de9 None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:28.049196 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02eb31f7-46c1-43b6-bcac-fbb813663de9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:28.049580 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02eb31f7-46c1-43b6-bcac-fbb813663de9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:28.051091 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02eb31f7-46c1-43b6-bcac-fbb813663de9 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:28.063568 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 482/978] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Aug 6 01:21:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:28.070577 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d3112e-4876-4e79-a286-ffa56ce3f952 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:28.175595 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d3112e-4876-4e79-a286-ffa56ce3f952 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:28.209692 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0911802a-1823-45fd-9668-35d7820fd62b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:28.210403 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0911802a-1823-45fd-9668-35d7820fd62b None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:28.210758 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0911802a-1823-45fd-9668-35d7820fd62b None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:28.211105 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0911802a-1823-45fd-9668-35d7820fd62b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:28.211957 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0911802a-1823-45fd-9668-35d7820fd62b None admin] RBAC: Adding query filter params (role.id=f42c599286be4e0fa632dc6769f95753, scope.project.id=d9d68d310bf24449a5fe20d2dc264a58, user.id=8e86f38590824894a3b960e6774ff8a9) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:28.212342 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0911802a-1823-45fd-9668-35d7820fd62b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:28.213980 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0911802a-1823-45fd-9668-35d7820fd62b None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:28.229090 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 497/979] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Aug 6 01:21:28 2022] GET /identity/v3/role_assignments?user.id=8e86f38590824894a3b960e6774ff8a9&scope.project.id=d9d68d310bf24449a5fe20d2dc264a58&role.id=f42c599286be4e0fa632dc6769f95753 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:21:28.576684 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a629c16-8cb7-4122-9746-fa0e23862339 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:28.577217 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a629c16-8cb7-4122-9746-fa0e23862339 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:28.577293 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a629c16-8cb7-4122-9746-fa0e23862339 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:28.578671 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 483/980] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:21:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:21:28.583751 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492e56ad-2060-46e2-b54c-1626b52fdcf0 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:28.584138 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492e56ad-2060-46e2-b54c-1626b52fdcf0 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:28.584322 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492e56ad-2060-46e2-b54c-1626b52fdcf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:28.602692 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-492e56ad-2060-46e2-b54c-1626b52fdcf0 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:21:28.604421 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-492e56ad-2060-46e2-b54c-1626b52fdcf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:21:28.630856 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:28.673882 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 498/981] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:21:28 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:21:28.696503 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a629c16-8cb7-4122-9746-fa0e23862339 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:28.800809 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a629c16-8cb7-4122-9746-fa0e23862339 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:28.834238 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8756e11d-2ced-410a-a042-c847844c7308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:28.834871 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8756e11d-2ced-410a-a042-c847844c7308 None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:28.835158 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8756e11d-2ced-410a-a042-c847844c7308 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:28.835441 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8756e11d-2ced-410a-a042-c847844c7308 None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:28.836145 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8756e11d-2ced-410a-a042-c847844c7308 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:28.837553 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8756e11d-2ced-410a-a042-c847844c7308 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:28.853397 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 484/982] 10.222.0.26 () {60 vars in 1141 bytes} [Sat Aug 6 01:21:28 2022] POST /identity/v3/services => generated 242 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:21:29.282086 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cab152-887f-49ec-82a5-a02e1fce9452 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:29.282420 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cab152-887f-49ec-82a5-a02e1fce9452 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:29.282653 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cab152-887f-49ec-82a5-a02e1fce9452 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:29.284144 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 499/983] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:21:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:21:29.289195 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff00e218-3c34-4608-a632-f913b6c9930b None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:29.289492 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff00e218-3c34-4608-a632-f913b6c9930b None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:29.289774 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff00e218-3c34-4608-a632-f913b6c9930b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:29.310713 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-ff00e218-3c34-4608-a632-f913b6c9930b None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:21:29.312253 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-ff00e218-3c34-4608-a632-f913b6c9930b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:21:29.362064 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:29.397100 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 485/984] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:21:29 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:21:29.418005 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cab152-887f-49ec-82a5-a02e1fce9452 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:29.520920 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cab152-887f-49ec-82a5-a02e1fce9452 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:29.554211 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a603d2-b335-4eda-8a40-07ef1ad88aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:29.554771 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a603d2-b335-4eda-8a40-07ef1ad88aca None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:29.555110 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a603d2-b335-4eda-8a40-07ef1ad88aca None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:29.555387 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a603d2-b335-4eda-8a40-07ef1ad88aca None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:29.556039 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a603d2-b335-4eda-8a40-07ef1ad88aca None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:29.557963 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a603d2-b335-4eda-8a40-07ef1ad88aca None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:29.566827 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-77a603d2-b335-4eda-8a40-07ef1ad88aca None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 06 01:21:29.566827 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:29.566827 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 01:21:29.566827 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 01:21:29.566827 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 01:21:29.566827 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 01:21:29.566827 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 01:21:29.566827 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 01:21:29.566827 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 06 01:21:29.566827 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:21:29.566827 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 01:21:29.566827 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:21:29.566827 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:29.566827 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:21:29.566827 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:21:29.566827 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:21:29.566827 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:21:29.567792 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:21:29.567792 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:21:29.567792 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:21:29.567792 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:21:29.567792 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:21:29.567792 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:21:29.567792 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 01:21:29.567792 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 01:21:29.567792 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 01:21:29.567792 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 01:21:29.567792 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:21:29.567792 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:21:29.567792 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:21:29.567792 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:21:29.567792 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:21:29.567792 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:21:29.568660 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:21:29.568660 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:21:29.568660 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:21:29.568660 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:21:29.568660 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:21:29.568660 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:21:29.568660 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:21:29.568660 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:21:29.568660 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:21:29.568660 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:21:29.568660 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 01:21:29.568660 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 01:21:29.568660 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 06 01:21:29.568660 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:21:29.570181 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 500/985] 10.222.0.26 () {56 vars in 1126 bytes} [Sat Aug 6 01:21:29 2022] GET /identity/v3/services/object-store => generated 93 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:21:29.574705 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff00e218-3c34-4608-a632-f913b6c9930b None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:29.678858 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff00e218-3c34-4608-a632-f913b6c9930b None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:29.715075 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddf25ca-af57-4b42-b5d3-8dc86eded6a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:29.715692 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddf25ca-af57-4b42-b5d3-8dc86eded6a8 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:29.715972 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddf25ca-af57-4b42-b5d3-8dc86eded6a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:29.716244 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddf25ca-af57-4b42-b5d3-8dc86eded6a8 None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:29.716915 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddf25ca-af57-4b42-b5d3-8dc86eded6a8 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:29.717244 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddf25ca-af57-4b42-b5d3-8dc86eded6a8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:29.718591 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddf25ca-af57-4b42-b5d3-8dc86eded6a8 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:29.728296 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 486/986] 10.222.0.26 () {56 vars in 1122 bytes} [Sat Aug 6 01:21:29 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:29.730713 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a603d2-b335-4eda-8a40-07ef1ad88aca None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:29.758864 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:29.832364 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a603d2-b335-4eda-8a40-07ef1ad88aca None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:29.866577 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2069aabf-90e4-400d-ba7d-2f6f44cf7e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:29.867190 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2069aabf-90e4-400d-ba7d-2f6f44cf7e1e None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:29.867470 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2069aabf-90e4-400d-ba7d-2f6f44cf7e1e None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:29.867739 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2069aabf-90e4-400d-ba7d-2f6f44cf7e1e None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:29.868404 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2069aabf-90e4-400d-ba7d-2f6f44cf7e1e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:29.868704 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2069aabf-90e4-400d-ba7d-2f6f44cf7e1e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:29.870435 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2069aabf-90e4-400d-ba7d-2f6f44cf7e1e None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:29.880008 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 501/987] 10.222.0.26 () {56 vars in 1122 bytes} [Sat Aug 6 01:21:29 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:29.881222 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddf25ca-af57-4b42-b5d3-8dc86eded6a8 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:29.923669 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:29.984568 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddf25ca-af57-4b42-b5d3-8dc86eded6a8 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:30.019343 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0db5fb-dc61-4ad6-8e92-ca476484c54c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:30.019961 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0db5fb-dc61-4ad6-8e92-ca476484c54c None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:30.020239 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0db5fb-dc61-4ad6-8e92-ca476484c54c None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:30.020515 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0db5fb-dc61-4ad6-8e92-ca476484c54c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:30.021283 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0db5fb-dc61-4ad6-8e92-ca476484c54c None admin] RBAC: Adding query filter params (interface=admin, service_id=c12c62708a51468e84396829b97276dd, region_id=RegionOne) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:30.021594 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0db5fb-dc61-4ad6-8e92-ca476484c54c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:30.022939 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0db5fb-dc61-4ad6-8e92-ca476484c54c None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:30.034006 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 487/988] 10.222.0.26 () {56 vars in 1249 bytes} [Sat Aug 6 01:21:29 2022] GET /identity/v3/endpoints?service_id=c12c62708a51468e84396829b97276dd&interface=admin®ion_id=RegionOne => generated 193 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:21:30.034177 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2069aabf-90e4-400d-ba7d-2f6f44cf7e1e None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:30.128785 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:30.139628 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2069aabf-90e4-400d-ba7d-2f6f44cf7e1e None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:30.173566 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff743ae-705a-4ea1-a934-fd74fc2f891b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:30.174193 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff743ae-705a-4ea1-a934-fd74fc2f891b None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:30.174474 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff743ae-705a-4ea1-a934-fd74fc2f891b None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:30.174746 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff743ae-705a-4ea1-a934-fd74fc2f891b None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:30.175387 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff743ae-705a-4ea1-a934-fd74fc2f891b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:30.176749 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff743ae-705a-4ea1-a934-fd74fc2f891b None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:30.183592 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 502/989] 10.222.0.26 () {56 vars in 1087 bytes} [Sat Aug 6 01:21:30 2022] GET /identity/v3/services => generated 1727 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:21:30.303622 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b21c5a-fbc3-4d20-9af2-75c740afbbf5 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:30.303964 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b21c5a-fbc3-4d20-9af2-75c740afbbf5 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:30.304195 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b21c5a-fbc3-4d20-9af2-75c740afbbf5 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:30.305789 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 488/990] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:21:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:21:30.318624 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600435e5-1466-44c5-9a5d-92efcfdd1006 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:30.318950 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600435e5-1466-44c5-9a5d-92efcfdd1006 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:30.319176 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600435e5-1466-44c5-9a5d-92efcfdd1006 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:30.339000 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-600435e5-1466-44c5-9a5d-92efcfdd1006 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:21:30.340213 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-600435e5-1466-44c5-9a5d-92efcfdd1006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:21:30.422538 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 503/991] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:21:30 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:21:30.444696 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b21c5a-fbc3-4d20-9af2-75c740afbbf5 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:30.550540 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b21c5a-fbc3-4d20-9af2-75c740afbbf5 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:30.552954 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:30.586482 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bf1cf5-3cdd-49f7-ac60-aeec4f9ee424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:30.587135 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bf1cf5-3cdd-49f7-ac60-aeec4f9ee424 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:30.587421 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bf1cf5-3cdd-49f7-ac60-aeec4f9ee424 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:30.587702 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bf1cf5-3cdd-49f7-ac60-aeec4f9ee424 None admin] PATH_INFO: `/v3/services/image` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:30.589163 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bf1cf5-3cdd-49f7-ac60-aeec4f9ee424 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:30.590536 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bf1cf5-3cdd-49f7-ac60-aeec4f9ee424 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:30.599631 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-f1bf1cf5-3cdd-49f7-ac60-aeec4f9ee424 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 06 01:21:30.599631 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:30.599631 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 01:21:30.599631 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 01:21:30.599631 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 01:21:30.599631 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 01:21:30.599631 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 01:21:30.599631 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 01:21:30.599631 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 06 01:21:30.599631 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:21:30.599631 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 01:21:30.599631 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:21:30.599631 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:30.599631 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:21:30.599631 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:21:30.599631 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:21:30.599631 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:21:30.600588 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:21:30.600588 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:21:30.600588 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:21:30.600588 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:21:30.600588 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:21:30.600588 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:21:30.600588 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 01:21:30.600588 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 01:21:30.600588 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 01:21:30.600588 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 01:21:30.600588 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:21:30.600588 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:21:30.600588 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:21:30.600588 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:21:30.600588 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:21:30.600588 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:21:30.602078 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:21:30.602078 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:21:30.602078 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:21:30.602078 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:21:30.602078 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:21:30.602078 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:21:30.602078 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:21:30.602078 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:21:30.602078 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:21:30.602078 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:21:30.602078 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 01:21:30.602078 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 01:21:30.602078 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 06 01:21:30.602078 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:21:30.603164 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 489/992] 10.222.0.26 () {56 vars in 1105 bytes} [Sat Aug 6 01:21:30 2022] GET /identity/v3/services/image => generated 86 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:21:30.607950 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600435e5-1466-44c5-9a5d-92efcfdd1006 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:30.695206 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:30.713567 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600435e5-1466-44c5-9a5d-92efcfdd1006 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:30.748165 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0518fe96-d4bd-4447-a4ae-81449494b7d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:30.748787 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0518fe96-d4bd-4447-a4ae-81449494b7d8 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:30.749072 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0518fe96-d4bd-4447-a4ae-81449494b7d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:30.749367 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0518fe96-d4bd-4447-a4ae-81449494b7d8 None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:30.750047 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0518fe96-d4bd-4447-a4ae-81449494b7d8 None admin] RBAC: Adding query filter params (name=image) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:30.750351 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0518fe96-d4bd-4447-a4ae-81449494b7d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:30.751719 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0518fe96-d4bd-4447-a4ae-81449494b7d8 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:30.761573 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 504/993] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:21:30 2022] GET /identity/v3/services?name=image => generated 122 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:30.762761 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bf1cf5-3cdd-49f7-ac60-aeec4f9ee424 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:30.866553 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bf1cf5-3cdd-49f7-ac60-aeec4f9ee424 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:30.882430 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7bdb0b-14b3-4a2e-a2e8-c74f92ed9d94 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:30.882710 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7bdb0b-14b3-4a2e-a2e8-c74f92ed9d94 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:30.882989 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7bdb0b-14b3-4a2e-a2e8-c74f92ed9d94 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:30.884644 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 505/994] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:21:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 01:21:30.889430 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4385dc8b-0fdd-4a78-82c9-96184203f428 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:30.889721 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4385dc8b-0fdd-4a78-82c9-96184203f428 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:30.889998 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4385dc8b-0fdd-4a78-82c9-96184203f428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:30.903273 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13662ebe-9767-453f-b8d4-57574cdaf500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:30.903897 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13662ebe-9767-453f-b8d4-57574cdaf500 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:30.904183 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13662ebe-9767-453f-b8d4-57574cdaf500 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:30.904460 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13662ebe-9767-453f-b8d4-57574cdaf500 None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:30.905167 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13662ebe-9767-453f-b8d4-57574cdaf500 None admin] RBAC: Adding query filter params (type=image) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:30.905477 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13662ebe-9767-453f-b8d4-57574cdaf500 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:30.906841 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13662ebe-9767-453f-b8d4-57574cdaf500 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:30.908369 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-4385dc8b-0fdd-4a78-82c9-96184203f428 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:21:30.909590 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-4385dc8b-0fdd-4a78-82c9-96184203f428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:21:30.911830 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 490/995] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:21:30 2022] GET /identity/v3/services?type=image => generated 122 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:30.986405 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 506/996] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:21:30 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 06 01:21:31.009967 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13662ebe-9767-453f-b8d4-57574cdaf500 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:31.044113 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:31.123179 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13662ebe-9767-453f-b8d4-57574cdaf500 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:31.164670 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5379fad-63e7-4f1c-8bd2-c0f4ac7eb8c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:31.165343 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5379fad-63e7-4f1c-8bd2-c0f4ac7eb8c9 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:31.165624 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5379fad-63e7-4f1c-8bd2-c0f4ac7eb8c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:31.165983 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5379fad-63e7-4f1c-8bd2-c0f4ac7eb8c9 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:31.167510 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5379fad-63e7-4f1c-8bd2-c0f4ac7eb8c9 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:31.168889 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5379fad-63e7-4f1c-8bd2-c0f4ac7eb8c9 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:31.177721 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-b5379fad-63e7-4f1c-8bd2-c0f4ac7eb8c9 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 06 01:21:31.177721 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:31.177721 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 01:21:31.177721 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 01:21:31.177721 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 01:21:31.177721 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 01:21:31.177721 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 01:21:31.177721 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 01:21:31.177721 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 06 01:21:31.177721 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:21:31.177721 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 01:21:31.177721 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:21:31.177721 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:31.177721 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:21:31.177721 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:21:31.177721 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:21:31.177721 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:21:31.178949 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:21:31.178949 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:21:31.178949 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:21:31.178949 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:21:31.178949 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:21:31.178949 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:21:31.178949 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 01:21:31.178949 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 01:21:31.178949 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 01:21:31.178949 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 01:21:31.178949 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:21:31.178949 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:21:31.178949 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:21:31.178949 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:21:31.178949 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:21:31.178949 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:21:31.180023 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:21:31.180023 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:21:31.180023 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:21:31.180023 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:21:31.180023 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:21:31.180023 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:21:31.180023 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:21:31.180023 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:21:31.180023 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:21:31.180023 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:21:31.180023 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 01:21:31.180023 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 01:21:31.180023 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 06 01:21:31.180023 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:21:31.184652 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 491/997] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Aug 6 01:21:31 2022] GET /identity/v3/services/compute => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:21:31.186024 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4385dc8b-0fdd-4a78-82c9-96184203f428 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:31.267552 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:31.290287 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4385dc8b-0fdd-4a78-82c9-96184203f428 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:31.327388 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500e254e-dbf4-4226-bea0-93fb4445c6b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:31.328010 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500e254e-dbf4-4226-bea0-93fb4445c6b5 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:31.328291 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500e254e-dbf4-4226-bea0-93fb4445c6b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:31.328563 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500e254e-dbf4-4226-bea0-93fb4445c6b5 None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:31.329266 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500e254e-dbf4-4226-bea0-93fb4445c6b5 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:31.329570 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500e254e-dbf4-4226-bea0-93fb4445c6b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:31.330927 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500e254e-dbf4-4226-bea0-93fb4445c6b5 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:31.340326 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 507/998] 10.222.0.26 () {56 vars in 1112 bytes} [Sat Aug 6 01:21:31 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:31.342144 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5379fad-63e7-4f1c-8bd2-c0f4ac7eb8c9 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:31.448955 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:31.455758 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5379fad-63e7-4f1c-8bd2-c0f4ac7eb8c9 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:31.489673 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2e96ba-84b9-474c-8e99-e93b510aa57f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:31.490298 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2e96ba-84b9-474c-8e99-e93b510aa57f None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:31.490580 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2e96ba-84b9-474c-8e99-e93b510aa57f None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:31.490857 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2e96ba-84b9-474c-8e99-e93b510aa57f None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:31.491533 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2e96ba-84b9-474c-8e99-e93b510aa57f None admin] RBAC: Adding query filter params (type=compute) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:31.491838 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2e96ba-84b9-474c-8e99-e93b510aa57f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:31.493265 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2e96ba-84b9-474c-8e99-e93b510aa57f None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:31.498749 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 492/999] 10.222.0.26 () {56 vars in 1112 bytes} [Sat Aug 6 01:21:31 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:31.504444 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500e254e-dbf4-4226-bea0-93fb4445c6b5 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:31.605766 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500e254e-dbf4-4226-bea0-93fb4445c6b5 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:31.639842 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1538eb-411e-4a61-8974-09d506484ca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:31.640465 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1538eb-411e-4a61-8974-09d506484ca5 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:31.640748 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1538eb-411e-4a61-8974-09d506484ca5 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:31.641025 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1538eb-411e-4a61-8974-09d506484ca5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:31.641807 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1538eb-411e-4a61-8974-09d506484ca5 None admin] RBAC: Adding query filter params (interface=public, service_id=c49009acb02c4347a21e01c7f00e3764, region_id=RegionOne) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:31.642119 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1538eb-411e-4a61-8974-09d506484ca5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:31.643476 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1538eb-411e-4a61-8974-09d506484ca5 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:31.651856 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 508/1000] 10.222.0.26 () {56 vars in 1251 bytes} [Sat Aug 6 01:21:31 2022] GET /identity/v3/endpoints?service_id=c49009acb02c4347a21e01c7f00e3764&interface=public®ion_id=RegionOne => generated 194 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:31.657321 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2e96ba-84b9-474c-8e99-e93b510aa57f None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:31.759816 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2e96ba-84b9-474c-8e99-e93b510aa57f None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:31.793775 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2149a039-caa0-41d3-a911-64ae3b1f5b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:31.794501 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2149a039-caa0-41d3-a911-64ae3b1f5b97 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:31.794742 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2149a039-caa0-41d3-a911-64ae3b1f5b97 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:31.795021 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2149a039-caa0-41d3-a911-64ae3b1f5b97 None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:31.795669 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2149a039-caa0-41d3-a911-64ae3b1f5b97 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:31.797031 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2149a039-caa0-41d3-a911-64ae3b1f5b97 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:31.809222 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 493/1001] 10.222.0.26 () {56 vars in 1087 bytes} [Sat Aug 6 01:21:31 2022] GET /identity/v3/services => generated 1727 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:21:32.218144 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0391070-b975-4f97-a463-9e98936e6e12 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:32.218701 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0391070-b975-4f97-a463-9e98936e6e12 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:32.218701 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0391070-b975-4f97-a463-9e98936e6e12 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:32.220286 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 509/1002] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:21:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:21:32.225617 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7e17bb-bf5a-4669-ab34-1808b34b575b None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:32.225908 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7e17bb-bf5a-4669-ab34-1808b34b575b None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:32.226446 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7e17bb-bf5a-4669-ab34-1808b34b575b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:32.245324 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-fd7e17bb-bf5a-4669-ab34-1808b34b575b None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:21:32.246509 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-fd7e17bb-bf5a-4669-ab34-1808b34b575b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:21:32.266297 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:32.318381 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 494/1003] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:21:32 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:21:32.340867 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0391070-b975-4f97-a463-9e98936e6e12 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:32.350446 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:32.454451 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0391070-b975-4f97-a463-9e98936e6e12 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:32.488394 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bce9f7-5c1e-44dd-831f-f4e4e4fd0a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:32.488963 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bce9f7-5c1e-44dd-831f-f4e4e4fd0a77 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:32.489284 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bce9f7-5c1e-44dd-831f-f4e4e4fd0a77 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:32.489564 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bce9f7-5c1e-44dd-831f-f4e4e4fd0a77 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:32.490197 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bce9f7-5c1e-44dd-831f-f4e4e4fd0a77 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:32.491564 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bce9f7-5c1e-44dd-831f-f4e4e4fd0a77 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:32.501765 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-43bce9f7-5c1e-44dd-831f-f4e4e4fd0a77 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 06 01:21:32.501765 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:32.501765 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 01:21:32.501765 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 01:21:32.501765 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 01:21:32.501765 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 01:21:32.501765 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 01:21:32.501765 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 01:21:32.501765 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 06 01:21:32.501765 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:21:32.501765 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 01:21:32.501765 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:21:32.501765 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:32.501765 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:21:32.501765 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:21:32.501765 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:21:32.501765 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:21:32.502789 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:21:32.502789 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:21:32.502789 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:21:32.502789 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:21:32.502789 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:21:32.502789 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:21:32.502789 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 01:21:32.502789 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 01:21:32.502789 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 01:21:32.502789 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 01:21:32.502789 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:21:32.502789 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:21:32.502789 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:21:32.502789 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:21:32.502789 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:21:32.502789 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:21:32.503627 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:21:32.503627 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:21:32.503627 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:21:32.503627 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:21:32.503627 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:21:32.503627 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:21:32.503627 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:21:32.503627 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:21:32.503627 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:21:32.503627 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:21:32.503627 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 01:21:32.503627 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 01:21:32.503627 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 06 01:21:32.503627 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:21:32.505130 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 510/1004] 10.222.0.26 () {56 vars in 1126 bytes} [Sat Aug 6 01:21:32 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:21:32.509251 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7e17bb-bf5a-4669-ab34-1808b34b575b None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:32.608626 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7e17bb-bf5a-4669-ab34-1808b34b575b None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:32.642278 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a37ae2-a124-410d-9cd6-4e455b5eb113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:32.642900 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a37ae2-a124-410d-9cd6-4e455b5eb113 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:32.643185 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a37ae2-a124-410d-9cd6-4e455b5eb113 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:32.643461 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a37ae2-a124-410d-9cd6-4e455b5eb113 None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:32.644140 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a37ae2-a124-410d-9cd6-4e455b5eb113 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:32.644445 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a37ae2-a124-410d-9cd6-4e455b5eb113 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:32.645859 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a37ae2-a124-410d-9cd6-4e455b5eb113 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:32.651226 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 495/1005] 10.222.0.26 () {56 vars in 1122 bytes} [Sat Aug 6 01:21:32 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:32.657067 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bce9f7-5c1e-44dd-831f-f4e4e4fd0a77 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:32.731138 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:32.762715 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bce9f7-5c1e-44dd-831f-f4e4e4fd0a77 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:32.798519 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259d83c2-709f-4b09-a024-89fc286be494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:32.799150 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259d83c2-709f-4b09-a024-89fc286be494 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:32.799434 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259d83c2-709f-4b09-a024-89fc286be494 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:32.799711 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259d83c2-709f-4b09-a024-89fc286be494 None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:32.800392 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259d83c2-709f-4b09-a024-89fc286be494 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:32.800697 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259d83c2-709f-4b09-a024-89fc286be494 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:32.802161 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259d83c2-709f-4b09-a024-89fc286be494 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:32.807774 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 511/1006] 10.222.0.26 () {56 vars in 1122 bytes} [Sat Aug 6 01:21:32 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:32.813423 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a37ae2-a124-410d-9cd6-4e455b5eb113 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:32.833702 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:32.920291 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a37ae2-a124-410d-9cd6-4e455b5eb113 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:32.937847 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76310454-b12d-4020-9b17-ffbc18ee7705 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:32.938137 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76310454-b12d-4020-9b17-ffbc18ee7705 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:32.938413 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76310454-b12d-4020-9b17-ffbc18ee7705 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:32.939986 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 512/1007] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:21:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:21:32.944507 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e406f6ac-b417-4b90-b1ca-9fc123a1d1bf None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:32.944788 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e406f6ac-b417-4b90-b1ca-9fc123a1d1bf None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:32.945091 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e406f6ac-b417-4b90-b1ca-9fc123a1d1bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:32.958187 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494efef9-a000-47e7-aabf-d6fc556aa5e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:32.958816 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494efef9-a000-47e7-aabf-d6fc556aa5e7 None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:32.959097 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494efef9-a000-47e7-aabf-d6fc556aa5e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:32.959535 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494efef9-a000-47e7-aabf-d6fc556aa5e7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:32.960267 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494efef9-a000-47e7-aabf-d6fc556aa5e7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:32.961687 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494efef9-a000-47e7-aabf-d6fc556aa5e7 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:32.964316 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-e406f6ac-b417-4b90-b1ca-9fc123a1d1bf None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:21:32.965678 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-e406f6ac-b417-4b90-b1ca-9fc123a1d1bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:21:32.979833 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 496/1008] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:21:32 2022] POST /identity/v3/endpoints => generated 325 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:21:33.048764 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 513/1009] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:21:32 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 06 01:21:33.072471 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494efef9-a000-47e7-aabf-d6fc556aa5e7 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:33.159278 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:33.182804 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494efef9-a000-47e7-aabf-d6fc556aa5e7 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:33.218114 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e780979-9fea-4a3d-8604-fdbad6a04515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:33.218745 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e780979-9fea-4a3d-8604-fdbad6a04515 None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:33.219033 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e780979-9fea-4a3d-8604-fdbad6a04515 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:33.219312 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e780979-9fea-4a3d-8604-fdbad6a04515 None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:33.220010 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e780979-9fea-4a3d-8604-fdbad6a04515 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:33.221402 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e780979-9fea-4a3d-8604-fdbad6a04515 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:33.237178 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 497/1010] 10.222.0.26 () {60 vars in 1141 bytes} [Sat Aug 6 01:21:33 2022] POST /identity/v3/services => generated 242 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:21:33.839479 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1950f5c4-a6af-4581-bf2d-8abdb8a1d360 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:33.840001 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1950f5c4-a6af-4581-bf2d-8abdb8a1d360 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:33.840001 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1950f5c4-a6af-4581-bf2d-8abdb8a1d360 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:33.841417 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 514/1011] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:21:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:21:33.846390 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec58ad6b-5e53-4f0e-a6bc-c558192ecbc0 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:33.846762 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec58ad6b-5e53-4f0e-a6bc-c558192ecbc0 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:33.846942 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec58ad6b-5e53-4f0e-a6bc-c558192ecbc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:33.864878 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-ec58ad6b-5e53-4f0e-a6bc-c558192ecbc0 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:21:33.866078 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-ec58ad6b-5e53-4f0e-a6bc-c558192ecbc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:21:33.940796 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 498/1012] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:21:33 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:21:33.963134 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1950f5c4-a6af-4581-bf2d-8abdb8a1d360 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:34.023655 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:34.066311 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1950f5c4-a6af-4581-bf2d-8abdb8a1d360 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:34.102421 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18651757-0889-47bc-bc71-2e532f4ea885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:34.102977 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18651757-0889-47bc-bc71-2e532f4ea885 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:34.103281 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18651757-0889-47bc-bc71-2e532f4ea885 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:34.104083 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18651757-0889-47bc-bc71-2e532f4ea885 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:34.104863 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18651757-0889-47bc-bc71-2e532f4ea885 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:34.106244 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18651757-0889-47bc-bc71-2e532f4ea885 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:34.115293 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-18651757-0889-47bc-bc71-2e532f4ea885 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 06 01:21:34.115293 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:34.115293 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 01:21:34.115293 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 01:21:34.115293 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 01:21:34.115293 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 01:21:34.115293 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 01:21:34.115293 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 01:21:34.115293 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 06 01:21:34.115293 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:21:34.115293 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 01:21:34.115293 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:21:34.115293 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:34.115293 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:21:34.115293 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:21:34.115293 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:21:34.115293 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:21:34.116336 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:21:34.116336 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:21:34.116336 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:21:34.116336 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:21:34.116336 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:21:34.116336 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:21:34.116336 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 01:21:34.116336 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 01:21:34.116336 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 01:21:34.116336 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 01:21:34.116336 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:21:34.116336 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:21:34.116336 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:21:34.116336 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:21:34.116336 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:21:34.116336 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:21:34.117639 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:21:34.117639 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:21:34.117639 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:21:34.117639 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:21:34.117639 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:21:34.117639 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:21:34.117639 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:21:34.117639 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:21:34.117639 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:21:34.117639 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:21:34.117639 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 01:21:34.117639 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 01:21:34.117639 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 06 01:21:34.117639 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:21:34.121267 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 515/1013] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Aug 6 01:21:33 2022] GET /identity/v3/services/compute => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:21:34.123478 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec58ad6b-5e53-4f0e-a6bc-c558192ecbc0 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:34.187792 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:34.228036 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec58ad6b-5e53-4f0e-a6bc-c558192ecbc0 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:34.261866 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d13132-9bdc-479f-be3a-71014693843c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:34.262483 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d13132-9bdc-479f-be3a-71014693843c None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:34.262759 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d13132-9bdc-479f-be3a-71014693843c None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:34.263028 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d13132-9bdc-479f-be3a-71014693843c None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:34.263694 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d13132-9bdc-479f-be3a-71014693843c None admin] RBAC: Adding query filter params (name=compute) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:34.263994 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d13132-9bdc-479f-be3a-71014693843c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:34.265362 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d13132-9bdc-479f-be3a-71014693843c None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:34.270282 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 499/1014] 10.222.0.26 () {56 vars in 1112 bytes} [Sat Aug 6 01:21:34 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:34.276349 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18651757-0889-47bc-bc71-2e532f4ea885 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:34.378964 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18651757-0889-47bc-bc71-2e532f4ea885 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:34.408442 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:34.419899 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3766a4bd-7102-4d8f-bbfd-86109ef23dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:34.420520 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3766a4bd-7102-4d8f-bbfd-86109ef23dba None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:34.420803 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3766a4bd-7102-4d8f-bbfd-86109ef23dba None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:34.421108 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3766a4bd-7102-4d8f-bbfd-86109ef23dba None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:34.421799 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3766a4bd-7102-4d8f-bbfd-86109ef23dba None admin] RBAC: Adding query filter params (type=compute) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:34.422107 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3766a4bd-7102-4d8f-bbfd-86109ef23dba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:34.423500 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3766a4bd-7102-4d8f-bbfd-86109ef23dba None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:34.433068 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 516/1015] 10.222.0.26 () {56 vars in 1112 bytes} [Sat Aug 6 01:21:34 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:34.434329 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d13132-9bdc-479f-be3a-71014693843c None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:34.536795 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d13132-9bdc-479f-be3a-71014693843c None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:34.562678 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:34.571282 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dc5358-2774-4682-91ce-fd0fe115208c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:34.571904 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dc5358-2774-4682-91ce-fd0fe115208c None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:34.572755 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dc5358-2774-4682-91ce-fd0fe115208c None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:34.573033 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dc5358-2774-4682-91ce-fd0fe115208c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:34.573770 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dc5358-2774-4682-91ce-fd0fe115208c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:34.575199 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dc5358-2774-4682-91ce-fd0fe115208c None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:34.601330 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 500/1016] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:21:34 2022] POST /identity/v3/endpoints => generated 334 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:21:35.035521 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d898d7-f35a-42ff-a4db-1d3b02c38503 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:35.036043 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d898d7-f35a-42ff-a4db-1d3b02c38503 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:35.036043 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d898d7-f35a-42ff-a4db-1d3b02c38503 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:35.037572 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 517/1017] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:21:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:21:35.043262 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c2ca98-7359-49f9-a835-7b5a6c7713b6 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:35.043698 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c2ca98-7359-49f9-a835-7b5a6c7713b6 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:35.043844 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c2ca98-7359-49f9-a835-7b5a6c7713b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:35.062516 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-87c2ca98-7359-49f9-a835-7b5a6c7713b6 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:21:35.063715 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-87c2ca98-7359-49f9-a835-7b5a6c7713b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:21:35.139180 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 501/1018] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:21:35 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:21:35.163144 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d898d7-f35a-42ff-a4db-1d3b02c38503 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:35.266434 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d898d7-f35a-42ff-a4db-1d3b02c38503 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:35.278641 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dd7060-2339-45ec-9747-650c3110d303 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:35.278932 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dd7060-2339-45ec-9747-650c3110d303 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:35.279208 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dd7060-2339-45ec-9747-650c3110d303 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:35.280806 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 502/1019] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:21:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:21:35.285462 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1584761a-2fb3-4883-84b3-682df3acfc31 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:35.285800 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1584761a-2fb3-4883-84b3-682df3acfc31 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:35.286020 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1584761a-2fb3-4883-84b3-682df3acfc31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:35.301910 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbed9799-7adc-4c33-9de8-c36f2dca09da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:35.302550 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbed9799-7adc-4c33-9de8-c36f2dca09da None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:35.302829 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbed9799-7adc-4c33-9de8-c36f2dca09da None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:35.303103 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbed9799-7adc-4c33-9de8-c36f2dca09da None admin] PATH_INFO: `/v3/domains/default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:35.304210 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-1584761a-2fb3-4883-84b3-682df3acfc31 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:21:35.304326 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbed9799-7adc-4c33-9de8-c36f2dca09da None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:35.305412 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-1584761a-2fb3-4883-84b3-682df3acfc31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:21:35.305692 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbed9799-7adc-4c33-9de8-c36f2dca09da None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:35.308153 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 518/1020] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:21:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:35.313485 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbed9799-7adc-4c33-9de8-c36f2dca09da None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:35.377784 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:35.379030 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 503/1021] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:21:35 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 06 01:21:35.401074 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1584761a-2fb3-4883-84b3-682df3acfc31 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:35.437323 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbed9799-7adc-4c33-9de8-c36f2dca09da None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:35.444947 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:35.479467 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698ad169-fea6-42f4-8ad6-389e9064d2da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:35.480223 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698ad169-fea6-42f4-8ad6-389e9064d2da None admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:35.480520 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698ad169-fea6-42f4-8ad6-389e9064d2da None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:35.481087 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698ad169-fea6-42f4-8ad6-389e9064d2da None admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:35.481512 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698ad169-fea6-42f4-8ad6-389e9064d2da None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:35.482959 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698ad169-fea6-42f4-8ad6-389e9064d2da None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:35.507436 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1584761a-2fb3-4883-84b3-682df3acfc31 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:35.510640 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 519/1022] 10.222.0.26 () {60 vars in 1141 bytes} [Sat Aug 6 01:21:35 2022] POST /identity/v3/projects => generated 311 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:21:35.546993 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96e1f14-968f-4308-8c84-ec04818d3b71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:35.547572 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96e1f14-968f-4308-8c84-ec04818d3b71 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:35.547866 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96e1f14-968f-4308-8c84-ec04818d3b71 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:35.548186 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96e1f14-968f-4308-8c84-ec04818d3b71 None admin] PATH_INFO: `/v3/services/image` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:35.548833 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96e1f14-968f-4308-8c84-ec04818d3b71 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:35.550252 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96e1f14-968f-4308-8c84-ec04818d3b71 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:35.560596 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-d96e1f14-968f-4308-8c84-ec04818d3b71 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 06 01:21:35.560596 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:35.560596 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 01:21:35.560596 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 01:21:35.560596 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 01:21:35.560596 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 01:21:35.560596 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 01:21:35.560596 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 01:21:35.560596 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 06 01:21:35.560596 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:21:35.560596 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 01:21:35.560596 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:21:35.560596 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:35.560596 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:21:35.560596 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:21:35.560596 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:21:35.560596 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:21:35.561700 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:21:35.561700 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:21:35.561700 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:21:35.561700 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:21:35.561700 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:21:35.561700 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:21:35.561700 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 01:21:35.561700 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 01:21:35.561700 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 01:21:35.561700 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 01:21:35.561700 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:21:35.561700 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:21:35.561700 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:21:35.561700 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:21:35.561700 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:21:35.561700 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:21:35.568091 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:21:35.568091 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:21:35.568091 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:21:35.568091 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:21:35.568091 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:21:35.568091 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:21:35.568091 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:21:35.568091 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:21:35.568091 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:21:35.568091 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:21:35.568091 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 01:21:35.568091 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 01:21:35.568091 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 06 01:21:35.568091 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:21:35.568091 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 504/1023] 10.222.0.26 () {56 vars in 1105 bytes} [Sat Aug 6 01:21:35 2022] GET /identity/v3/services/image => generated 86 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:21:35.568882 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698ad169-fea6-42f4-8ad6-389e9064d2da None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:35.685332 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698ad169-fea6-42f4-8ad6-389e9064d2da None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:35.695565 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:35.726999 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793e9492-7721-41e2-a21a-80d804d2f59e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:35.727616 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793e9492-7721-41e2-a21a-80d804d2f59e None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:35.727893 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793e9492-7721-41e2-a21a-80d804d2f59e None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:35.728285 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793e9492-7721-41e2-a21a-80d804d2f59e None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:35.729633 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793e9492-7721-41e2-a21a-80d804d2f59e None admin] RBAC: Adding query filter params (name=image) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:35.729961 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793e9492-7721-41e2-a21a-80d804d2f59e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:35.731292 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793e9492-7721-41e2-a21a-80d804d2f59e None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:35.736487 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 520/1024] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:21:35 2022] GET /identity/v3/services?name=image => generated 122 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:35.742373 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96e1f14-968f-4308-8c84-ec04818d3b71 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:35.849504 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96e1f14-968f-4308-8c84-ec04818d3b71 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:35.852050 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:35.886313 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c596c9d2-f3c0-4d2a-a8a8-6dac4cfd613c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:35.887011 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c596c9d2-f3c0-4d2a-a8a8-6dac4cfd613c None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:35.887298 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c596c9d2-f3c0-4d2a-a8a8-6dac4cfd613c None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:35.887578 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c596c9d2-f3c0-4d2a-a8a8-6dac4cfd613c None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:35.888259 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c596c9d2-f3c0-4d2a-a8a8-6dac4cfd613c None admin] RBAC: Adding query filter params (type=image) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:35.888569 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c596c9d2-f3c0-4d2a-a8a8-6dac4cfd613c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:35.889985 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c596c9d2-f3c0-4d2a-a8a8-6dac4cfd613c None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:35.896351 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 505/1025] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:21:35 2022] GET /identity/v3/services?type=image => generated 350 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:35.901861 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793e9492-7721-41e2-a21a-80d804d2f59e None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:36.005889 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793e9492-7721-41e2-a21a-80d804d2f59e None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:36.040334 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620945aa-69ae-4f99-980f-ca00265db6a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:36.040973 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620945aa-69ae-4f99-980f-ca00265db6a0 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:36.041282 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620945aa-69ae-4f99-980f-ca00265db6a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:36.041576 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620945aa-69ae-4f99-980f-ca00265db6a0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:36.042317 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620945aa-69ae-4f99-980f-ca00265db6a0 None admin] RBAC: Adding query filter params (interface=public, service_id=482436591eef42aab1e635ad0f342fff, region_id=RegionOne) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:36.042628 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620945aa-69ae-4f99-980f-ca00265db6a0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:36.044000 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620945aa-69ae-4f99-980f-ca00265db6a0 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:36.050615 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 521/1026] 10.222.0.26 () {56 vars in 1251 bytes} [Sat Aug 6 01:21:35 2022] GET /identity/v3/endpoints?service_id=482436591eef42aab1e635ad0f342fff&interface=public®ion_id=RegionOne => generated 194 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:36.056344 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c596c9d2-f3c0-4d2a-a8a8-6dac4cfd613c None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:36.161408 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c596c9d2-f3c0-4d2a-a8a8-6dac4cfd613c None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:36.195803 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b03590-7022-410c-9bcf-7e9b024bc006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:36.196421 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b03590-7022-410c-9bcf-7e9b024bc006 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:36.196702 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b03590-7022-410c-9bcf-7e9b024bc006 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:36.197070 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b03590-7022-410c-9bcf-7e9b024bc006 None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:36.197904 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b03590-7022-410c-9bcf-7e9b024bc006 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:36.199256 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b03590-7022-410c-9bcf-7e9b024bc006 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:36.206105 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 506/1027] 10.222.0.26 () {56 vars in 1087 bytes} [Sat Aug 6 01:21:36 2022] GET /identity/v3/services => generated 1957 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:21:36.722574 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc24422a-7ed3-4be7-9712-faf00a71da6c None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:36.722944 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc24422a-7ed3-4be7-9712-faf00a71da6c None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:36.723152 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc24422a-7ed3-4be7-9712-faf00a71da6c None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:36.724722 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 522/1028] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:21:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:21:36.730795 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd2ce07-9e53-46c1-ae0a-3762aa76ea3e None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:36.731138 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd2ce07-9e53-46c1-ae0a-3762aa76ea3e None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:36.731376 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd2ce07-9e53-46c1-ae0a-3762aa76ea3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:36.749560 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-8bd2ce07-9e53-46c1-ae0a-3762aa76ea3e None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:21:36.750762 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-8bd2ce07-9e53-46c1-ae0a-3762aa76ea3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:21:36.777855 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:36.820370 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 507/1029] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:21:36 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:21:36.844807 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc24422a-7ed3-4be7-9712-faf00a71da6c None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:36.882935 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:36.950791 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc24422a-7ed3-4be7-9712-faf00a71da6c None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:36.984511 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d5e355-4e8a-49f0-8361-6449b646be48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:36.985099 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d5e355-4e8a-49f0-8361-6449b646be48 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:36.985386 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d5e355-4e8a-49f0-8361-6449b646be48 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:36.985663 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d5e355-4e8a-49f0-8361-6449b646be48 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:36.996257 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d5e355-4e8a-49f0-8361-6449b646be48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:21:37.001185 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-40d5e355-4e8a-49f0-8361-6449b646be48 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 06 01:21:37.001185 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:37.001185 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:21:37.001185 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:21:37.001185 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:21:37.001185 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:21:37.001185 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:21:37.001185 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:21:37.001185 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:21:37.001185 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:21:37.001185 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:21:37.001185 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:21:37.001185 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:21:37.001185 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:21:37.001185 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:21:37.001185 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:21:37.002090 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:21:37.002090 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:21:37.002090 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:21:37.002090 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:21:37.002090 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:21:37.002090 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:21:37.002090 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:21:37.002090 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:21:37.002090 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:21:37.002090 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:21:37.002090 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:21:37.002090 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:21:37.002090 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:21:37.002090 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:21:37.002090 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:21:37.002090 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:21:37.002973 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:21:37.002973 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:21:37.002973 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:21:37.002973 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:21:37.002973 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:21:37.002973 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:21:37.002973 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:21:37.002973 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:21:37.002973 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:21:37.002973 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:21:37.002973 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 06 01:21:37.002973 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:21:37.002973 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 523/1030] 10.222.0.26 () {56 vars in 1120 bytes} [Sat Aug 6 01:21:36 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:21:37.008465 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd2ce07-9e53-46c1-ae0a-3762aa76ea3e None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:37.110626 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd2ce07-9e53-46c1-ae0a-3762aa76ea3e None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:37.143998 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c24964-c152-407f-b18e-739be2391dee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:37.144621 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c24964-c152-407f-b18e-739be2391dee None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:37.144910 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c24964-c152-407f-b18e-739be2391dee None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:37.145217 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c24964-c152-407f-b18e-739be2391dee None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:37.145942 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c24964-c152-407f-b18e-739be2391dee None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:37.146249 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c24964-c152-407f-b18e-739be2391dee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:37.147608 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c24964-c152-407f-b18e-739be2391dee None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:37.162461 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 508/1031] 10.222.0.26 () {56 vars in 1115 bytes} [Sat Aug 6 01:21:37 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:37.169283 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d5e355-4e8a-49f0-8361-6449b646be48 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:37.267437 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:37.275596 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d5e355-4e8a-49f0-8361-6449b646be48 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:37.311170 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be4d1a5-6fd5-4a4e-b18d-cfe1f8709167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:37.311845 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be4d1a5-6fd5-4a4e-b18d-cfe1f8709167 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:37.312130 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be4d1a5-6fd5-4a4e-b18d-cfe1f8709167 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:37.312412 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be4d1a5-6fd5-4a4e-b18d-cfe1f8709167 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:37.324994 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be4d1a5-6fd5-4a4e-b18d-cfe1f8709167 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:37.326506 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be4d1a5-6fd5-4a4e-b18d-cfe1f8709167 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:37.342547 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-0be4d1a5-6fd5-4a4e-b18d-cfe1f8709167 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:21:37.342547 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:37.342547 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:21:37.342547 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:21:37.342547 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:21:37.342547 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:21:37.342547 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:21:37.342547 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:21:37.342547 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:21:37.342547 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:21:37.342547 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:21:37.342547 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:21:37.342547 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:21:37.342547 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:21:37.342547 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:21:37.343522 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:21:37.343522 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:21:37.343522 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:21:37.343522 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:21:37.343522 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:21:37.343522 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:21:37.343522 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:21:37.343522 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:21:37.343522 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:21:37.343522 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:21:37.343522 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:21:37.343522 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:21:37.343522 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:21:37.343522 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:21:37.343522 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:21:37.343522 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:21:37.343522 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:21:37.344456 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:21:37.344456 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:21:37.344456 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:21:37.344456 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:21:37.345366 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 524/1032] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:21:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:21:37.349860 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c24964-c152-407f-b18e-739be2391dee None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:37.377682 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:37.462041 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c24964-c152-407f-b18e-739be2391dee None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:37.497022 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7375fd-7183-40d2-a93b-e95e9812f24b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:37.497676 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7375fd-7183-40d2-a93b-e95e9812f24b None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:37.497960 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7375fd-7183-40d2-a93b-e95e9812f24b None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:37.498236 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7375fd-7183-40d2-a93b-e95e9812f24b None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:37.498920 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7375fd-7183-40d2-a93b-e95e9812f24b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:37.499227 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7375fd-7183-40d2-a93b-e95e9812f24b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:37.500573 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7375fd-7183-40d2-a93b-e95e9812f24b None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:37.517365 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 509/1033] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Aug 6 01:21:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:37.518976 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be4d1a5-6fd5-4a4e-b18d-cfe1f8709167 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:37.625588 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be4d1a5-6fd5-4a4e-b18d-cfe1f8709167 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:37.639702 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74e4722-e389-4d5f-9f86-73aaf6bcbd5f None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:37.639990 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74e4722-e389-4d5f-9f86-73aaf6bcbd5f None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:37.640257 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74e4722-e389-4d5f-9f86-73aaf6bcbd5f None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:37.641889 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 510/1034] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:21:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:21:37.648235 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2acbc9c-6cbf-4218-b60e-e42bac1ae970 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:37.648528 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2acbc9c-6cbf-4218-b60e-e42bac1ae970 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:37.648810 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2acbc9c-6cbf-4218-b60e-e42bac1ae970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:37.666325 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2797be-4d3a-4d69-9c6a-5e3896b97403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:37.667106 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2797be-4d3a-4d69-9c6a-5e3896b97403 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:37.667388 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2797be-4d3a-4d69-9c6a-5e3896b97403 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:37.667662 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2797be-4d3a-4d69-9c6a-5e3896b97403 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:37.667820 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-f2acbc9c-6cbf-4218-b60e-e42bac1ae970 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:21:37.668973 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-f2acbc9c-6cbf-4218-b60e-e42bac1ae970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:21:37.675275 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2797be-4d3a-4d69-9c6a-5e3896b97403 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:37.676768 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2797be-4d3a-4d69-9c6a-5e3896b97403 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:37.688716 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-7c2797be-4d3a-4d69-9c6a-5e3896b97403 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 06 01:21:37.688716 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:37.688716 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:21:37.688716 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:21:37.688716 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:21:37.688716 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:21:37.688716 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:21:37.688716 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:21:37.688716 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:21:37.688716 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:21:37.688716 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:21:37.688716 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:21:37.688716 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 01:21:37.688716 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 01:21:37.688716 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 01:21:37.688716 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 01:21:37.689843 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:21:37.689843 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:21:37.689843 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 01:21:37.689843 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:21:37.689843 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 01:21:37.689843 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:21:37.689843 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:21:37.689843 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:21:37.689843 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:21:37.689843 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:21:37.689843 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:21:37.689843 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:21:37.689843 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:21:37.689843 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:21:37.689843 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:21:37.689843 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:21:37.689843 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:21:37.689843 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:21:37.690781 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:21:37.690781 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:21:37.690781 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 01:21:37.690781 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 01:21:37.690781 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 01:21:37.690781 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 01:21:37.690781 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 01:21:37.690781 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 01:21:37.690781 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 06 01:21:37.690781 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:21:37.694597 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 525/1035] 10.222.0.26 () {56 vars in 1093 bytes} [Sat Aug 6 01:21:37 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:21:37.695679 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2797be-4d3a-4d69-9c6a-5e3896b97403 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:37.707731 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:37.745594 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 511/1036] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:21:37 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 06 01:21:37.770214 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2acbc9c-6cbf-4218-b60e-e42bac1ae970 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:37.785611 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:37.816060 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2797be-4d3a-4d69-9c6a-5e3896b97403 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:37.853874 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bab6cb-311f-4228-a77b-42ef5ef2341f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:37.854759 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bab6cb-311f-4228-a77b-42ef5ef2341f None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:37.855524 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bab6cb-311f-4228-a77b-42ef5ef2341f None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:37.855875 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bab6cb-311f-4228-a77b-42ef5ef2341f None admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:37.856733 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bab6cb-311f-4228-a77b-42ef5ef2341f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:37.857128 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bab6cb-311f-4228-a77b-42ef5ef2341f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:37.859282 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bab6cb-311f-4228-a77b-42ef5ef2341f None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:37.882984 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 526/1037] 10.222.0.26 () {56 vars in 1133 bytes} [Sat Aug 6 01:21:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:21:37.883995 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bab6cb-311f-4228-a77b-42ef5ef2341f None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:37.888070 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2acbc9c-6cbf-4218-b60e-e42bac1ae970 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:37.925784 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107efaa9-8e8c-456f-977c-20246ae39d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:37.926430 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107efaa9-8e8c-456f-977c-20246ae39d53 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:37.926710 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107efaa9-8e8c-456f-977c-20246ae39d53 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:37.926984 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107efaa9-8e8c-456f-977c-20246ae39d53 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:37.928138 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107efaa9-8e8c-456f-977c-20246ae39d53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:37.929613 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107efaa9-8e8c-456f-977c-20246ae39d53 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:37.931915 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 512/1038] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:21:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:21:37.937242 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107efaa9-8e8c-456f-977c-20246ae39d53 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:37.961555 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:38.001769 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bab6cb-311f-4228-a77b-42ef5ef2341f None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:38.026503 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:38.039968 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e788dc5e-10ad-4307-8ffd-6322a0d4ccc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:38.040636 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e788dc5e-10ad-4307-8ffd-6322a0d4ccc7 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:38.041564 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e788dc5e-10ad-4307-8ffd-6322a0d4ccc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:38.041846 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e788dc5e-10ad-4307-8ffd-6322a0d4ccc7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:38.045985 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107efaa9-8e8c-456f-977c-20246ae39d53 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:38.080255 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07ecce0-7c5a-411b-a9e2-662873a3b2f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:38.080963 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07ecce0-7c5a-411b-a9e2-662873a3b2f0 None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:38.081344 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07ecce0-7c5a-411b-a9e2-662873a3b2f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:38.081717 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07ecce0-7c5a-411b-a9e2-662873a3b2f0 None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:38.082483 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07ecce0-7c5a-411b-a9e2-662873a3b2f0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:38.083971 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07ecce0-7c5a-411b-a9e2-662873a3b2f0 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:38.117078 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 513/1039] 10.222.0.26 () {60 vars in 1132 bytes} [Sat Aug 6 01:21:37 2022] POST /identity/v3/users => generated 285 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:21:38.117709 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e788dc5e-10ad-4307-8ffd-6322a0d4ccc7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:38.117709 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e788dc5e-10ad-4307-8ffd-6322a0d4ccc7 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:38.189201 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:38.242444 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 527/1040] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Aug 6 01:21:37 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 06 01:21:38.244678 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07ecce0-7c5a-411b-a9e2-662873a3b2f0 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:38.245709 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e140212-7556-4ca2-a0d3-8bde8e519c4d None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:38.246004 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e140212-7556-4ca2-a0d3-8bde8e519c4d None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:38.246282 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e140212-7556-4ca2-a0d3-8bde8e519c4d None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:38.247646 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 528/1041] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:21:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:21:38.253084 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c7c5d0-95b3-45b8-bc26-1384aa0423ff None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:38.253372 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c7c5d0-95b3-45b8-bc26-1384aa0423ff None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:38.253650 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c7c5d0-95b3-45b8-bc26-1384aa0423ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:38.272947 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-a5c7c5d0-95b3-45b8-bc26-1384aa0423ff None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:21:38.274169 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-a5c7c5d0-95b3-45b8-bc26-1384aa0423ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:21:38.357786 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 529/1042] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:21:38 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 06 01:21:38.360792 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07ecce0-7c5a-411b-a9e2-662873a3b2f0 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:38.380972 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c7c5d0-95b3-45b8-bc26-1384aa0423ff None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:38.400644 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0474011-aeba-43ab-b15e-d04bb695cb41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:38.401354 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0474011-aeba-43ab-b15e-d04bb695cb41 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:38.401637 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0474011-aeba-43ab-b15e-d04bb695cb41 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:38.404795 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0474011-aeba-43ab-b15e-d04bb695cb41 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:38.411882 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:38.418473 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0474011-aeba-43ab-b15e-d04bb695cb41 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:38.419989 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0474011-aeba-43ab-b15e-d04bb695cb41 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:38.436129 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-a0474011-aeba-43ab-b15e-d04bb695cb41 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:21:38.436129 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:38.436129 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:21:38.436129 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:21:38.436129 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:21:38.436129 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:21:38.436129 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:21:38.436129 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:21:38.436129 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:21:38.436129 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:21:38.436129 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:21:38.436129 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:21:38.436129 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:21:38.436129 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:21:38.436129 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:21:38.437201 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:21:38.437201 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:21:38.437201 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:21:38.437201 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:21:38.437201 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:21:38.437201 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:21:38.437201 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:21:38.437201 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:21:38.437201 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:21:38.437201 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:21:38.437201 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:21:38.437201 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:21:38.437201 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:21:38.437201 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:21:38.437201 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:21:38.437201 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:21:38.437201 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:21:38.443255 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:21:38.443255 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:21:38.443255 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:21:38.443255 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:21:38.443255 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 514/1043] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:21:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:21:38.445021 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0474011-aeba-43ab-b15e-d04bb695cb41 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:38.486876 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:38.493066 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c7c5d0-95b3-45b8-bc26-1384aa0423ff None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:38.532139 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720a95e5-07b9-439d-ba81-a99eac866a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:38.532793 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720a95e5-07b9-439d-ba81-a99eac866a94 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:38.533101 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720a95e5-07b9-439d-ba81-a99eac866a94 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:38.533380 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720a95e5-07b9-439d-ba81-a99eac866a94 None admin] PATH_INFO: `/v3/services/image` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:38.534020 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720a95e5-07b9-439d-ba81-a99eac866a94 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:38.535384 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720a95e5-07b9-439d-ba81-a99eac866a94 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:38.544516 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-720a95e5-07b9-439d-ba81-a99eac866a94 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 06 01:21:38.544516 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:38.544516 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 01:21:38.544516 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 01:21:38.544516 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 01:21:38.544516 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 01:21:38.544516 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 01:21:38.544516 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 01:21:38.544516 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 06 01:21:38.544516 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:21:38.544516 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 01:21:38.544516 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:21:38.544516 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:38.544516 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:21:38.544516 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:21:38.544516 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:21:38.544516 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:21:38.545553 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:21:38.545553 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:21:38.545553 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:21:38.545553 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:21:38.545553 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:21:38.545553 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:21:38.545553 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 01:21:38.545553 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 01:21:38.545553 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 01:21:38.545553 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 01:21:38.545553 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:21:38.545553 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:21:38.545553 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:21:38.545553 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:21:38.545553 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:21:38.545553 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:21:38.546385 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:21:38.546385 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:21:38.546385 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:21:38.546385 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:21:38.546385 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:21:38.546385 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:21:38.546385 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:21:38.546385 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:21:38.546385 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:21:38.546385 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:21:38.546385 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 01:21:38.546385 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 01:21:38.546385 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 06 01:21:38.546385 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:21:38.546385 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 530/1044] 10.222.0.26 () {56 vars in 1105 bytes} [Sat Aug 6 01:21:38 2022] GET /identity/v3/services/image => generated 86 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:21:38.551661 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720a95e5-07b9-439d-ba81-a99eac866a94 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:38.558299 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0474011-aeba-43ab-b15e-d04bb695cb41 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:38.594842 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c5e3a1-dcea-402a-ba57-6c33bb6aafd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:38.595466 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c5e3a1-dcea-402a-ba57-6c33bb6aafd1 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:38.595750 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c5e3a1-dcea-402a-ba57-6c33bb6aafd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:38.596051 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c5e3a1-dcea-402a-ba57-6c33bb6aafd1 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:38.596870 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c5e3a1-dcea-402a-ba57-6c33bb6aafd1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:38.597211 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c5e3a1-dcea-402a-ba57-6c33bb6aafd1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:38.598573 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c5e3a1-dcea-402a-ba57-6c33bb6aafd1 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:38.611091 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 515/1045] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Aug 6 01:21:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:21:38.616513 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c5e3a1-dcea-402a-ba57-6c33bb6aafd1 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:38.668891 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:38.672378 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720a95e5-07b9-439d-ba81-a99eac866a94 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:38.708120 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba94888-0acd-4960-959a-0129e1361f45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:38.708752 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba94888-0acd-4960-959a-0129e1361f45 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:38.709036 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba94888-0acd-4960-959a-0129e1361f45 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:38.709339 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba94888-0acd-4960-959a-0129e1361f45 None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:38.710019 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba94888-0acd-4960-959a-0129e1361f45 None admin] RBAC: Adding query filter params (name=image) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:38.710326 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba94888-0acd-4960-959a-0129e1361f45 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:38.711688 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba94888-0acd-4960-959a-0129e1361f45 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:38.717229 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 531/1046] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:21:38 2022] GET /identity/v3/services?name=image => generated 122 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:21:38.723238 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba94888-0acd-4960-959a-0129e1361f45 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:38.733677 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c5e3a1-dcea-402a-ba57-6c33bb6aafd1 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:38.749132 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:38.773763 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044e14f7-7cdb-4fed-844a-64bf73e14351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:38.774399 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044e14f7-7cdb-4fed-844a-64bf73e14351 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:38.774685 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044e14f7-7cdb-4fed-844a-64bf73e14351 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:38.774995 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044e14f7-7cdb-4fed-844a-64bf73e14351 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:38.783358 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044e14f7-7cdb-4fed-844a-64bf73e14351 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:38.785088 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044e14f7-7cdb-4fed-844a-64bf73e14351 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:38.797882 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-044e14f7-7cdb-4fed-844a-64bf73e14351 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:21:38.797882 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:38.797882 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:21:38.797882 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:21:38.797882 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:21:38.797882 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:21:38.797882 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:21:38.797882 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:21:38.797882 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:21:38.797882 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:21:38.797882 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:21:38.797882 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:21:38.797882 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 01:21:38.797882 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 01:21:38.797882 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 01:21:38.798795 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 01:21:38.798795 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:21:38.798795 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:21:38.798795 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:21:38.798795 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:21:38.798795 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:21:38.798795 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:21:38.798795 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:21:38.798795 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:21:38.798795 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:21:38.798795 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:21:38.798795 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:21:38.798795 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:21:38.798795 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:21:38.798795 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:21:38.798795 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:21:38.798795 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:21:38.799679 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 01:21:38.799679 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 01:21:38.799679 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 01:21:38.799679 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 01:21:38.799679 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 01:21:38.799679 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 01:21:38.799679 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:21:38.799679 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:21:38.799679 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 516/1047] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Aug 6 01:21:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:21:38.804733 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044e14f7-7cdb-4fed-844a-64bf73e14351 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:38.833591 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba94888-0acd-4960-959a-0129e1361f45 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:38.871455 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f441e7a4-f0e1-4487-80ae-2396141b2c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:38.872114 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f441e7a4-f0e1-4487-80ae-2396141b2c22 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:38.872397 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f441e7a4-f0e1-4487-80ae-2396141b2c22 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:38.872801 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f441e7a4-f0e1-4487-80ae-2396141b2c22 None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:38.873496 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f441e7a4-f0e1-4487-80ae-2396141b2c22 None admin] RBAC: Adding query filter params (type=image) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:38.873803 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f441e7a4-f0e1-4487-80ae-2396141b2c22 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:38.875170 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f441e7a4-f0e1-4487-80ae-2396141b2c22 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:38.885955 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 532/1048] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:21:38 2022] GET /identity/v3/services?type=image => generated 350 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:21:38.888156 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f441e7a4-f0e1-4487-80ae-2396141b2c22 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:38.919424 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044e14f7-7cdb-4fed-844a-64bf73e14351 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:38.930470 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:38.958406 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c213639-35f0-4901-8ff8-dcb40138a140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:38.959112 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c213639-35f0-4901-8ff8-dcb40138a140 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:38.959400 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c213639-35f0-4901-8ff8-dcb40138a140 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:38.959676 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c213639-35f0-4901-8ff8-dcb40138a140 None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:38.960595 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c213639-35f0-4901-8ff8-dcb40138a140 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:38.961070 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c213639-35f0-4901-8ff8-dcb40138a140 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:38.962519 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c213639-35f0-4901-8ff8-dcb40138a140 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:38.975951 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 517/1049] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Aug 6 01:21:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:21:38.982195 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c213639-35f0-4901-8ff8-dcb40138a140 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:38.986609 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:39.000515 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f441e7a4-f0e1-4487-80ae-2396141b2c22 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:39.035907 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6f6686-7d03-417a-a2df-b7f4240da11f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:39.036518 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6f6686-7d03-417a-a2df-b7f4240da11f None admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:39.036799 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6f6686-7d03-417a-a2df-b7f4240da11f None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:39.037092 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6f6686-7d03-417a-a2df-b7f4240da11f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:39.037802 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6f6686-7d03-417a-a2df-b7f4240da11f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:39.039154 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6f6686-7d03-417a-a2df-b7f4240da11f None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:39.060806 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 533/1050] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:21:38 2022] POST /identity/v3/endpoints => generated 327 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:21:39.099467 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c213639-35f0-4901-8ff8-dcb40138a140 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:39.144264 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f668fd-673b-4327-b7ba-6c563aa2baa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:39.144921 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f668fd-673b-4327-b7ba-6c563aa2baa6 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:39.145188 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f668fd-673b-4327-b7ba-6c563aa2baa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:39.145385 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f668fd-673b-4327-b7ba-6c563aa2baa6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:39.146181 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f668fd-673b-4327-b7ba-6c563aa2baa6 None admin] RBAC: Adding query filter params (role.id=f42c599286be4e0fa632dc6769f95753, scope.project.id=d9d68d310bf24449a5fe20d2dc264a58, user.id=b87adc1d5bc240e99f95f6595b350be4) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:39.146492 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f668fd-673b-4327-b7ba-6c563aa2baa6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:39.147961 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f668fd-673b-4327-b7ba-6c563aa2baa6 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:39.161093 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 518/1051] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Aug 6 01:21:38 2022] GET /identity/v3/role_assignments?user.id=b87adc1d5bc240e99f95f6595b350be4&scope.project.id=d9d68d310bf24449a5fe20d2dc264a58&role.id=f42c599286be4e0fa632dc6769f95753 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:21:40.181664 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb9b07c-6def-466e-8658-ad851198533d None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:40.182170 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb9b07c-6def-466e-8658-ad851198533d None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:40.182421 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb9b07c-6def-466e-8658-ad851198533d None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:40.183753 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 534/1052] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:21:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:21:40.188881 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2050b8-f475-43bd-b578-af5b7d62f225 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:40.189331 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2050b8-f475-43bd-b578-af5b7d62f225 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:40.189495 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2050b8-f475-43bd-b578-af5b7d62f225 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:40.208331 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-db2050b8-f475-43bd-b578-af5b7d62f225 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:21:40.209675 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-db2050b8-f475-43bd-b578-af5b7d62f225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:21:40.237283 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:40.289096 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 519/1053] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:21:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:21:40.307907 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb9b07c-6def-466e-8658-ad851198533d None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:40.416775 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb9b07c-6def-466e-8658-ad851198533d None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:40.450561 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1234221d-98b6-4f44-b4a3-f713bf8b4248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:40.451451 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1234221d-98b6-4f44-b4a3-f713bf8b4248 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:40.451933 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1234221d-98b6-4f44-b4a3-f713bf8b4248 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:40.452407 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1234221d-98b6-4f44-b4a3-f713bf8b4248 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:40.463078 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1234221d-98b6-4f44-b4a3-f713bf8b4248 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:21:40.468165 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-1234221d-98b6-4f44-b4a3-f713bf8b4248 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:21:40.468165 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:40.468165 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:21:40.468165 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:21:40.468165 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:21:40.468165 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:21:40.468165 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:21:40.468165 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:21:40.468165 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:21:40.468165 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:21:40.468165 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:21:40.468165 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:21:40.468165 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:21:40.468165 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:21:40.468165 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:21:40.468165 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:21:40.469170 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:21:40.469170 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:21:40.469170 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:21:40.469170 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:21:40.469170 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:21:40.469170 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:21:40.469170 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:21:40.469170 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:21:40.469170 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:21:40.469170 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:21:40.469170 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:21:40.469170 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:21:40.469170 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:21:40.469170 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:21:40.469170 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:21:40.469170 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:21:40.470025 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:21:40.470025 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:21:40.470025 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:21:40.470025 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:21:40.470025 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:21:40.470025 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:21:40.470025 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:21:40.470025 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:21:40.470025 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:21:40.470025 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:21:40.470025 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:21:40.470025 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:21:40.473183 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 535/1054] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Aug 6 01:21:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:21:40.476592 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2050b8-f475-43bd-b578-af5b7d62f225 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:40.576831 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2050b8-f475-43bd-b578-af5b7d62f225 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:40.611178 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2159dcc1-6d6e-40be-88a3-b8842745b329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:40.611825 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2159dcc1-6d6e-40be-88a3-b8842745b329 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:40.612110 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2159dcc1-6d6e-40be-88a3-b8842745b329 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:40.612388 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2159dcc1-6d6e-40be-88a3-b8842745b329 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:40.613132 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2159dcc1-6d6e-40be-88a3-b8842745b329 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:40.613443 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2159dcc1-6d6e-40be-88a3-b8842745b329 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:40.614820 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2159dcc1-6d6e-40be-88a3-b8842745b329 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:40.627869 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 520/1055] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Aug 6 01:21:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:40.629507 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1234221d-98b6-4f44-b4a3-f713bf8b4248 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:40.635631 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:40.736656 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1234221d-98b6-4f44-b4a3-f713bf8b4248 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:40.771674 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952f8fb4-c1e3-40f0-b66b-5cddf8c54724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:40.772318 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952f8fb4-c1e3-40f0-b66b-5cddf8c54724 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:40.772602 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952f8fb4-c1e3-40f0-b66b-5cddf8c54724 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:40.772880 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952f8fb4-c1e3-40f0-b66b-5cddf8c54724 None admin] PATH_INFO: `/v3/users/27aee98548fc43e5bdfc329dd2adf33c` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:40.786688 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952f8fb4-c1e3-40f0-b66b-5cddf8c54724 None admin] RBAC: Authorizing `identity:get_user(user_id=27aee98548fc43e5bdfc329dd2adf33c)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:40.788139 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952f8fb4-c1e3-40f0-b66b-5cddf8c54724 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:40.790003 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 536/1056] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Aug 6 01:21:40 2022] GET /identity/v3/users/27aee98548fc43e5bdfc329dd2adf33c => generated 285 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:40.795805 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2159dcc1-6d6e-40be-88a3-b8842745b329 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:40.823023 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:40.901572 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2159dcc1-6d6e-40be-88a3-b8842745b329 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:40.935240 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6dea06-800d-4bc4-8d25-b91e0d85445b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:40.935860 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6dea06-800d-4bc4-8d25-b91e0d85445b None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:40.936144 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6dea06-800d-4bc4-8d25-b91e0d85445b None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:40.936425 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6dea06-800d-4bc4-8d25-b91e0d85445b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:41.004705 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6dea06-800d-4bc4-8d25-b91e0d85445b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:41.006118 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6dea06-800d-4bc4-8d25-b91e0d85445b None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:41.031284 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:41.114593 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 521/1057] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Aug 6 01:21:40 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 06 01:21:41.116401 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952f8fb4-c1e3-40f0-b66b-5cddf8c54724 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:41.126407 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c169d466-ecb8-4a50-98cc-4b59e3be09e9 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:41.126704 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c169d466-ecb8-4a50-98cc-4b59e3be09e9 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:41.126988 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c169d466-ecb8-4a50-98cc-4b59e3be09e9 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:41.128484 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 522/1058] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:21:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 01:21:41.133123 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7b78f8-e3d1-45d5-9e3a-f5f45d771ae6 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:41.133471 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7b78f8-e3d1-45d5-9e3a-f5f45d771ae6 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:41.133678 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7b78f8-e3d1-45d5-9e3a-f5f45d771ae6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:41.152346 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-dd7b78f8-e3d1-45d5-9e3a-f5f45d771ae6 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:21:41.153581 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-dd7b78f8-e3d1-45d5-9e3a-f5f45d771ae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:21:41.181465 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:41.220519 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 523/1059] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:21:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 06 01:21:41.221195 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952f8fb4-c1e3-40f0-b66b-5cddf8c54724 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:41.221822 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1e8f9e-c9b7-414b-a6dd-48f7ef0a6992 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:41.222110 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1e8f9e-c9b7-414b-a6dd-48f7ef0a6992 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:41.222389 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1e8f9e-c9b7-414b-a6dd-48f7ef0a6992 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:41.224109 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 524/1060] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:21:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:21:41.229600 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9198f24a-2b8d-4aea-aae6-aa8dbaf3612d None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:41.229887 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9198f24a-2b8d-4aea-aae6-aa8dbaf3612d None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:41.230168 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9198f24a-2b8d-4aea-aae6-aa8dbaf3612d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:41.250024 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-9198f24a-2b8d-4aea-aae6-aa8dbaf3612d None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:21:41.251221 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-9198f24a-2b8d-4aea-aae6-aa8dbaf3612d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:21:41.258323 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd73aa6e-f216-4df1-914c-b8acd0d68e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:41.258903 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd73aa6e-f216-4df1-914c-b8acd0d68e96 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:41.259185 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd73aa6e-f216-4df1-914c-b8acd0d68e96 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:41.260010 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd73aa6e-f216-4df1-914c-b8acd0d68e96 None admin] PATH_INFO: `/v3/projects/36b6ddf715a747079ae3b654c838f46f` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:41.261281 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd73aa6e-f216-4df1-914c-b8acd0d68e96 None admin] RBAC: Authorizing `identity:get_project(project_id=36b6ddf715a747079ae3b654c838f46f)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:41.262709 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd73aa6e-f216-4df1-914c-b8acd0d68e96 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:41.264367 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 537/1061] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Aug 6 01:21:41 2022] GET /identity/v3/projects/36b6ddf715a747079ae3b654c838f46f => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:21:41.265137 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd73aa6e-f216-4df1-914c-b8acd0d68e96 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:41.303764 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:41.325739 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 525/1062] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:21:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 06 01:21:41.326583 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9198f24a-2b8d-4aea-aae6-aa8dbaf3612d None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:41.366668 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd73aa6e-f216-4df1-914c-b8acd0d68e96 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:41.394972 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:41.401557 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9f9e76-95cf-400d-ae48-ce3bd9be1724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:41.402163 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9f9e76-95cf-400d-ae48-ce3bd9be1724 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:41.402443 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9f9e76-95cf-400d-ae48-ce3bd9be1724 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:41.402719 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9f9e76-95cf-400d-ae48-ce3bd9be1724 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:41.437851 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9198f24a-2b8d-4aea-aae6-aa8dbaf3612d None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:41.469288 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9f9e76-95cf-400d-ae48-ce3bd9be1724 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:41.470763 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9f9e76-95cf-400d-ae48-ce3bd9be1724 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:41.471007 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d4d594-9962-4524-9b60-d3c6a570da8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:41.471625 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d4d594-9962-4524-9b60-d3c6a570da8a None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:41.471902 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d4d594-9962-4524-9b60-d3c6a570da8a None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:41.472178 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d4d594-9962-4524-9b60-d3c6a570da8a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:41.472951 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d4d594-9962-4524-9b60-d3c6a570da8a None admin] RBAC: Adding query filter params (role.id=4559b9a0837249f895a13c70096f8d38, scope.project.id=36b6ddf715a747079ae3b654c838f46f, user.id=27aee98548fc43e5bdfc329dd2adf33c) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:41.473291 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d4d594-9962-4524-9b60-d3c6a570da8a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:41.474678 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d4d594-9962-4524-9b60-d3c6a570da8a None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:41.480779 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 526/1063] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Aug 6 01:21:41 2022] GET /identity/v3/role_assignments?user.id=27aee98548fc43e5bdfc329dd2adf33c&scope.project.id=36b6ddf715a747079ae3b654c838f46f&role.id=4559b9a0837249f895a13c70096f8d38 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:41.481590 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d4d594-9962-4524-9b60-d3c6a570da8a None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:41.549337 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:41.583928 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 538/1064] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Aug 6 01:21:41 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 06 01:21:41.590505 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9f9e76-95cf-400d-ae48-ce3bd9be1724 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:41.613381 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d4d594-9962-4524-9b60-d3c6a570da8a None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:41.625599 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:41.653015 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0fd56a-aa44-4438-9b73-6405f1589b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:41.653832 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0fd56a-aa44-4438-9b73-6405f1589b98 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:41.654123 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0fd56a-aa44-4438-9b73-6405f1589b98 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:41.654405 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0fd56a-aa44-4438-9b73-6405f1589b98 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:41.664951 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0fd56a-aa44-4438-9b73-6405f1589b98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:21:41.670403 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-7e0fd56a-aa44-4438-9b73-6405f1589b98 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 06 01:21:41.670403 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:41.670403 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:21:41.670403 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:21:41.670403 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:21:41.670403 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:21:41.670403 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:21:41.670403 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:21:41.670403 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:21:41.670403 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:21:41.670403 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:21:41.670403 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:21:41.670403 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:21:41.670403 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:21:41.670403 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:21:41.670403 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:21:41.671366 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:21:41.671366 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:21:41.671366 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:21:41.671366 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:21:41.671366 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:21:41.671366 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:21:41.671366 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:21:41.671366 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:21:41.671366 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:21:41.671366 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:21:41.671366 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:21:41.671366 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:21:41.671366 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:21:41.671366 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:21:41.671366 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:21:41.671366 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:21:41.672213 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:21:41.672213 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:21:41.672213 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:21:41.672213 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:21:41.672213 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:21:41.672213 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:21:41.672213 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:21:41.672213 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:21:41.672213 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:21:41.672213 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:21:41.672213 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 06 01:21:41.672213 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:21:41.672213 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 527/1065] 10.222.0.26 () {56 vars in 1120 bytes} [Sat Aug 6 01:21:41 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:21:41.678160 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0fd56a-aa44-4438-9b73-6405f1589b98 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:41.707100 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9f9e76-95cf-400d-ae48-ce3bd9be1724 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:41.745666 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0c978a-2691-4211-a66f-a84b34b3340d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:41.746310 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0c978a-2691-4211-a66f-a84b34b3340d None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:41.746594 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0c978a-2691-4211-a66f-a84b34b3340d None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:41.746868 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0c978a-2691-4211-a66f-a84b34b3340d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:41.757541 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0c978a-2691-4211-a66f-a84b34b3340d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:41.758984 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0c978a-2691-4211-a66f-a84b34b3340d None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:41.775805 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-3d0c978a-2691-4211-a66f-a84b34b3340d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:21:41.775805 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:41.775805 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:21:41.775805 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:21:41.775805 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:21:41.775805 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:21:41.775805 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:21:41.775805 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:21:41.775805 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:21:41.775805 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:21:41.775805 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:21:41.775805 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:21:41.775805 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:21:41.775805 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:21:41.775805 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:21:41.777015 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:21:41.777015 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:21:41.777015 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:21:41.777015 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:21:41.777015 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:21:41.777015 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:21:41.777015 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:21:41.777015 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:21:41.777015 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:21:41.777015 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:21:41.777015 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:21:41.777015 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:21:41.777015 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:21:41.777015 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:21:41.777015 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:21:41.777015 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:21:41.777015 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:21:41.777959 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:21:41.777959 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:21:41.777959 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:21:41.777959 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:21:41.777959 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 539/1066] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:21:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:21:41.782804 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0c978a-2691-4211-a66f-a84b34b3340d None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:41.787650 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0fd56a-aa44-4438-9b73-6405f1589b98 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:41.795420 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:41.824822 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970560fc-5487-4781-98ae-2353f5ccf082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:41.825450 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970560fc-5487-4781-98ae-2353f5ccf082 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:41.825732 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970560fc-5487-4781-98ae-2353f5ccf082 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:41.826006 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970560fc-5487-4781-98ae-2353f5ccf082 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:41.826710 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970560fc-5487-4781-98ae-2353f5ccf082 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:41.827014 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970560fc-5487-4781-98ae-2353f5ccf082 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:41.828361 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970560fc-5487-4781-98ae-2353f5ccf082 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:41.841385 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 528/1067] 10.222.0.26 () {56 vars in 1115 bytes} [Sat Aug 6 01:21:41 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:21:41.844055 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970560fc-5487-4781-98ae-2353f5ccf082 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:41.874613 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:41.901836 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0c978a-2691-4211-a66f-a84b34b3340d None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:41.937496 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b882962-409b-45de-abd1-82eb669763c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:41.938120 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b882962-409b-45de-abd1-82eb669763c4 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:41.938395 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b882962-409b-45de-abd1-82eb669763c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:41.938664 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b882962-409b-45de-abd1-82eb669763c4 None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:41.939357 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b882962-409b-45de-abd1-82eb669763c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:41.939658 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b882962-409b-45de-abd1-82eb669763c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:41.942427 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b882962-409b-45de-abd1-82eb669763c4 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:41.961093 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 540/1068] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Aug 6 01:21:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:21:41.981599 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970560fc-5487-4781-98ae-2353f5ccf082 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:42.020260 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3423ff72-1d79-4778-ba74-b96c036a6209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:42.020985 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3423ff72-1d79-4778-ba74-b96c036a6209 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:42.021306 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3423ff72-1d79-4778-ba74-b96c036a6209 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:42.021589 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3423ff72-1d79-4778-ba74-b96c036a6209 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:42.034067 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3423ff72-1d79-4778-ba74-b96c036a6209 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:42.035615 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3423ff72-1d79-4778-ba74-b96c036a6209 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:42.052061 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-3423ff72-1d79-4778-ba74-b96c036a6209 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:21:42.052061 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:42.052061 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:21:42.052061 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:21:42.052061 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:21:42.052061 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:21:42.052061 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:21:42.052061 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:21:42.052061 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:21:42.052061 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:21:42.052061 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:21:42.052061 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:21:42.052061 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:21:42.052061 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:21:42.052061 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:21:42.053037 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:21:42.053037 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:21:42.053037 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:21:42.053037 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:21:42.053037 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:21:42.053037 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:21:42.053037 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:21:42.053037 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:21:42.053037 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:21:42.053037 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:21:42.053037 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:21:42.053037 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:21:42.053037 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:21:42.053037 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:21:42.053037 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:21:42.053037 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:21:42.053037 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:21:42.053992 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:21:42.053992 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:21:42.053992 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:21:42.053992 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:21:42.053992 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 529/1069] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:21:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:21:42.060978 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b882962-409b-45de-abd1-82eb669763c4 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:42.168137 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b882962-409b-45de-abd1-82eb669763c4 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:42.207604 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf866fc-4a38-458d-966b-92149c2f7219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:42.208289 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf866fc-4a38-458d-966b-92149c2f7219 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:42.208576 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf866fc-4a38-458d-966b-92149c2f7219 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:42.208850 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf866fc-4a38-458d-966b-92149c2f7219 None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:42.210257 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf866fc-4a38-458d-966b-92149c2f7219 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:42.210257 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf866fc-4a38-458d-966b-92149c2f7219 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:42.211218 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf866fc-4a38-458d-966b-92149c2f7219 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:42.223294 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 541/1070] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Aug 6 01:21:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:42.230345 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3423ff72-1d79-4778-ba74-b96c036a6209 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:42.255939 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:42.331778 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3423ff72-1d79-4778-ba74-b96c036a6209 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:42.365107 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39daeaf7-747b-4067-b0a1-1e7505bb42d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:42.365737 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39daeaf7-747b-4067-b0a1-1e7505bb42d6 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:42.366017 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39daeaf7-747b-4067-b0a1-1e7505bb42d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:42.366292 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39daeaf7-747b-4067-b0a1-1e7505bb42d6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:42.373643 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39daeaf7-747b-4067-b0a1-1e7505bb42d6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:42.375056 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39daeaf7-747b-4067-b0a1-1e7505bb42d6 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:42.386062 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-39daeaf7-747b-4067-b0a1-1e7505bb42d6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 06 01:21:42.386062 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:42.386062 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:21:42.386062 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:21:42.386062 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:21:42.386062 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:21:42.386062 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:21:42.386062 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:21:42.386062 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:21:42.386062 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:21:42.386062 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:21:42.386062 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:21:42.386062 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 01:21:42.386062 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 01:21:42.386062 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 01:21:42.386062 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 01:21:42.386962 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:21:42.386962 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:21:42.386962 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 01:21:42.386962 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:21:42.386962 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 01:21:42.386962 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:21:42.386962 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:21:42.386962 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:21:42.386962 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:21:42.386962 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:21:42.386962 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:21:42.386962 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:21:42.386962 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:21:42.386962 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:21:42.386962 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:21:42.386962 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:21:42.386962 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:21:42.386962 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:21:42.387920 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:21:42.387920 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:21:42.387920 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 01:21:42.387920 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 01:21:42.387920 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 01:21:42.387920 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 01:21:42.387920 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 01:21:42.387920 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 01:21:42.387920 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 06 01:21:42.387920 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:21:42.387920 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 530/1071] 10.222.0.26 () {56 vars in 1093 bytes} [Sat Aug 6 01:21:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:21:42.393094 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf866fc-4a38-458d-966b-92149c2f7219 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:42.395791 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:42.492014 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf866fc-4a38-458d-966b-92149c2f7219 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:42.523857 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79782716-5170-45c7-a805-88577b2c3cc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:42.524466 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79782716-5170-45c7-a805-88577b2c3cc6 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:42.524746 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79782716-5170-45c7-a805-88577b2c3cc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:42.525666 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79782716-5170-45c7-a805-88577b2c3cc6 None admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:42.526592 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79782716-5170-45c7-a805-88577b2c3cc6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:42.526897 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79782716-5170-45c7-a805-88577b2c3cc6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:42.528319 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79782716-5170-45c7-a805-88577b2c3cc6 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:42.545839 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 542/1072] 10.222.0.26 () {56 vars in 1133 bytes} [Sat Aug 6 01:21:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:42.551376 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39daeaf7-747b-4067-b0a1-1e7505bb42d6 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:42.635254 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:42.655143 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39daeaf7-747b-4067-b0a1-1e7505bb42d6 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:42.688949 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e887461-761c-417e-8aed-771b51fbc1d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:42.689615 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e887461-761c-417e-8aed-771b51fbc1d7 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:42.689903 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e887461-761c-417e-8aed-771b51fbc1d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:42.690184 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e887461-761c-417e-8aed-771b51fbc1d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:42.700569 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e887461-761c-417e-8aed-771b51fbc1d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:42.702028 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e887461-761c-417e-8aed-771b51fbc1d7 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:42.716069 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-8e887461-761c-417e-8aed-771b51fbc1d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:21:42.716069 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:42.716069 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:21:42.716069 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:21:42.716069 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:21:42.716069 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:21:42.716069 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:21:42.716069 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:21:42.716069 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:21:42.716069 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:21:42.716069 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:21:42.716069 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:21:42.716069 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:21:42.716069 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:21:42.716069 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:21:42.717122 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:21:42.717122 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:21:42.717122 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:21:42.717122 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:21:42.717122 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:21:42.717122 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:21:42.717122 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:21:42.717122 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:21:42.717122 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:21:42.717122 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:21:42.717122 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:21:42.717122 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:21:42.717122 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:21:42.717122 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:21:42.717122 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:21:42.717122 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:21:42.717122 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:21:42.718051 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:21:42.718051 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:21:42.718051 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:21:42.718051 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:21:42.718051 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 531/1073] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:21:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:21:42.722949 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79782716-5170-45c7-a805-88577b2c3cc6 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:42.780743 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:42.820303 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79782716-5170-45c7-a805-88577b2c3cc6 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:42.852492 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0332d28-1472-4849-8923-a15219717a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:42.853739 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0332d28-1472-4849-8923-a15219717a26 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:42.854025 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0332d28-1472-4849-8923-a15219717a26 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:42.854298 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0332d28-1472-4849-8923-a15219717a26 None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:42.854975 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0332d28-1472-4849-8923-a15219717a26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:42.855275 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0332d28-1472-4849-8923-a15219717a26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:42.856621 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0332d28-1472-4849-8923-a15219717a26 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:42.867359 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 543/1074] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Aug 6 01:21:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:42.873600 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e887461-761c-417e-8aed-771b51fbc1d7 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:42.971173 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e887461-761c-417e-8aed-771b51fbc1d7 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:42.999905 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:43.005376 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be9ee95-43f3-455d-8ac5-b9ca521cdd32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:43.006012 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be9ee95-43f3-455d-8ac5-b9ca521cdd32 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:43.006290 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be9ee95-43f3-455d-8ac5-b9ca521cdd32 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:43.006560 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be9ee95-43f3-455d-8ac5-b9ca521cdd32 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:43.013245 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be9ee95-43f3-455d-8ac5-b9ca521cdd32 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:43.014678 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be9ee95-43f3-455d-8ac5-b9ca521cdd32 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:43.026784 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-2be9ee95-43f3-455d-8ac5-b9ca521cdd32 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:21:43.026784 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:43.026784 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:21:43.026784 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:21:43.026784 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:21:43.026784 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:21:43.026784 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:21:43.026784 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:21:43.026784 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:21:43.026784 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:21:43.026784 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:21:43.026784 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:21:43.026784 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 01:21:43.026784 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 01:21:43.026784 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 01:21:43.027662 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 01:21:43.027662 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:21:43.027662 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:21:43.027662 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:21:43.027662 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:21:43.027662 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:21:43.027662 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:21:43.027662 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:21:43.027662 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:21:43.027662 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:21:43.027662 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:21:43.027662 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:21:43.027662 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:21:43.027662 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:21:43.027662 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:21:43.027662 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:21:43.027662 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:21:43.028561 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 01:21:43.028561 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 01:21:43.028561 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 01:21:43.028561 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 01:21:43.028561 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 01:21:43.028561 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 01:21:43.028561 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:21:43.028561 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:21:43.028561 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 532/1075] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Aug 6 01:21:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:21:43.033983 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0332d28-1472-4849-8923-a15219717a26 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:43.134810 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0332d28-1472-4849-8923-a15219717a26 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:43.154360 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:43.169320 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff51a78-79d1-4521-ad2a-e6e9bc68e2cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:43.169944 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff51a78-79d1-4521-ad2a-e6e9bc68e2cc None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:43.170223 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff51a78-79d1-4521-ad2a-e6e9bc68e2cc None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:43.170492 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff51a78-79d1-4521-ad2a-e6e9bc68e2cc None admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:43.171232 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff51a78-79d1-4521-ad2a-e6e9bc68e2cc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:43.171533 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff51a78-79d1-4521-ad2a-e6e9bc68e2cc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:43.179002 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff51a78-79d1-4521-ad2a-e6e9bc68e2cc None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:43.191771 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 544/1076] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Aug 6 01:21:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:43.198682 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be9ee95-43f3-455d-8ac5-b9ca521cdd32 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:43.297564 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be9ee95-43f3-455d-8ac5-b9ca521cdd32 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:43.332814 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f00a0a0-4f9b-4f29-a30a-6b66d456bd7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:43.333508 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f00a0a0-4f9b-4f29-a30a-6b66d456bd7f None admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:43.333789 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f00a0a0-4f9b-4f29-a30a-6b66d456bd7f None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:43.334063 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f00a0a0-4f9b-4f29-a30a-6b66d456bd7f None admin] PATH_INFO: `/v3/projects/d9d68d310bf24449a5fe20d2dc264a58/users/b87adc1d5bc240e99f95f6595b350be4/roles/f42c599286be4e0fa632dc6769f95753` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:43.336217 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f00a0a0-4f9b-4f29-a30a-6b66d456bd7f None admin] RBAC: Authorizing `identity:create_grant(project_id=d9d68d310bf24449a5fe20d2dc264a58, user_id=b87adc1d5bc240e99f95f6595b350be4, role_id=f42c599286be4e0fa632dc6769f95753)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:43.338459 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f00a0a0-4f9b-4f29-a30a-6b66d456bd7f None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:43.357723 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 533/1077] 10.222.0.26 () {58 vars in 1439 bytes} [Sat Aug 6 01:21:43 2022] PUT /identity/v3/projects/d9d68d310bf24449a5fe20d2dc264a58/users/b87adc1d5bc240e99f95f6595b350be4/roles/f42c599286be4e0fa632dc6769f95753 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:21:43.516876 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe124f5-9d51-44fc-8221-7ce58247ed59 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:43.517419 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe124f5-9d51-44fc-8221-7ce58247ed59 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:43.517509 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe124f5-9d51-44fc-8221-7ce58247ed59 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:43.518802 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 545/1078] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:21:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:21:43.524381 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1446b50-d195-4967-bdf5-313b01a5f215 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:43.524742 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1446b50-d195-4967-bdf5-313b01a5f215 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:43.529984 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1446b50-d195-4967-bdf5-313b01a5f215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:43.543781 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-a1446b50-d195-4967-bdf5-313b01a5f215 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:21:43.544966 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-a1446b50-d195-4967-bdf5-313b01a5f215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:21:43.591452 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:43.612421 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 534/1079] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:21:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:21:43.635148 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe124f5-9d51-44fc-8221-7ce58247ed59 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:43.738372 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe124f5-9d51-44fc-8221-7ce58247ed59 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:43.771327 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a13b5e-5be9-42f0-8a71-9208eec5cd76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:43.771958 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a13b5e-5be9-42f0-8a71-9208eec5cd76 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:43.772233 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a13b5e-5be9-42f0-8a71-9208eec5cd76 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:43.772501 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a13b5e-5be9-42f0-8a71-9208eec5cd76 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:43.783245 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a13b5e-5be9-42f0-8a71-9208eec5cd76 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:21:43.788074 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-a2a13b5e-5be9-42f0-8a71-9208eec5cd76 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:21:43.788074 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:43.788074 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:21:43.788074 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:21:43.788074 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:21:43.788074 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:21:43.788074 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:21:43.788074 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:21:43.788074 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:21:43.788074 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:21:43.788074 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:21:43.788074 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:21:43.788074 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:21:43.788074 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:21:43.788074 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:21:43.788074 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:21:43.789116 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:21:43.789116 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:21:43.789116 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:21:43.789116 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:21:43.789116 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:21:43.789116 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:21:43.789116 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:21:43.789116 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:21:43.789116 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:21:43.789116 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:21:43.789116 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:21:43.789116 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:21:43.789116 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:21:43.789116 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:21:43.789116 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:21:43.789116 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:21:43.791058 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:21:43.791058 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:21:43.791058 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:21:43.791058 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:21:43.791058 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:21:43.791058 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:21:43.791058 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:21:43.791058 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:21:43.791058 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:21:43.791058 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:21:43.791058 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:21:43.791058 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:21:43.791058 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 546/1080] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Aug 6 01:21:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:21:43.796169 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1446b50-d195-4967-bdf5-313b01a5f215 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:43.892392 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1446b50-d195-4967-bdf5-313b01a5f215 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:43.924844 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133b8fdf-7545-4c1d-a877-63f0938a6167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:43.925477 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133b8fdf-7545-4c1d-a877-63f0938a6167 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:43.925760 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133b8fdf-7545-4c1d-a877-63f0938a6167 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:43.926035 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133b8fdf-7545-4c1d-a877-63f0938a6167 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:43.926740 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133b8fdf-7545-4c1d-a877-63f0938a6167 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:43.927045 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133b8fdf-7545-4c1d-a877-63f0938a6167 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:43.928391 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133b8fdf-7545-4c1d-a877-63f0938a6167 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:43.936783 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 535/1081] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Aug 6 01:21:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:43.942608 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a13b5e-5be9-42f0-8a71-9208eec5cd76 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:43.984879 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:44.043136 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a13b5e-5be9-42f0-8a71-9208eec5cd76 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:44.076712 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05861644-f869-43ab-a371-5bf09d595b25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:44.077413 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05861644-f869-43ab-a371-5bf09d595b25 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:44.077691 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05861644-f869-43ab-a371-5bf09d595b25 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:44.077963 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05861644-f869-43ab-a371-5bf09d595b25 None admin] PATH_INFO: `/v3/users/27aee98548fc43e5bdfc329dd2adf33c` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:44.079195 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05861644-f869-43ab-a371-5bf09d595b25 None admin] RBAC: Authorizing `identity:get_user(user_id=27aee98548fc43e5bdfc329dd2adf33c)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:44.080635 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05861644-f869-43ab-a371-5bf09d595b25 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:44.082357 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 547/1082] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Aug 6 01:21:43 2022] GET /identity/v3/users/27aee98548fc43e5bdfc329dd2adf33c => generated 285 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:44.088773 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133b8fdf-7545-4c1d-a877-63f0938a6167 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:44.133918 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:44.186540 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133b8fdf-7545-4c1d-a877-63f0938a6167 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:44.218753 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c454a1bc-12c4-49b0-92a5-063768df42c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:44.219389 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c454a1bc-12c4-49b0-92a5-063768df42c6 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:44.219673 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c454a1bc-12c4-49b0-92a5-063768df42c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:44.219951 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c454a1bc-12c4-49b0-92a5-063768df42c6 None admin] PATH_INFO: `/v3/projects/36b6ddf715a747079ae3b654c838f46f` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:44.221120 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c454a1bc-12c4-49b0-92a5-063768df42c6 None admin] RBAC: Authorizing `identity:get_project(project_id=36b6ddf715a747079ae3b654c838f46f)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:44.222545 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c454a1bc-12c4-49b0-92a5-063768df42c6 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:44.224356 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 536/1083] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Aug 6 01:21:44 2022] GET /identity/v3/projects/36b6ddf715a747079ae3b654c838f46f => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:44.230882 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05861644-f869-43ab-a371-5bf09d595b25 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:44.329031 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05861644-f869-43ab-a371-5bf09d595b25 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:44.338944 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:44.363914 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7ffe04-9af3-4a9d-8e44-e606c4388662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:44.364581 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7ffe04-9af3-4a9d-8e44-e606c4388662 None admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:44.364864 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7ffe04-9af3-4a9d-8e44-e606c4388662 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:44.365241 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7ffe04-9af3-4a9d-8e44-e606c4388662 None admin] PATH_INFO: `/v3/projects/36b6ddf715a747079ae3b654c838f46f/users/27aee98548fc43e5bdfc329dd2adf33c/roles/4559b9a0837249f895a13c70096f8d38` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:44.367181 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7ffe04-9af3-4a9d-8e44-e606c4388662 None admin] RBAC: Authorizing `identity:create_grant(project_id=36b6ddf715a747079ae3b654c838f46f, user_id=27aee98548fc43e5bdfc329dd2adf33c, role_id=4559b9a0837249f895a13c70096f8d38)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:44.368656 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7ffe04-9af3-4a9d-8e44-e606c4388662 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:44.381663 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 548/1084] 10.222.0.26 () {58 vars in 1439 bytes} [Sat Aug 6 01:21:44 2022] PUT /identity/v3/projects/36b6ddf715a747079ae3b654c838f46f/users/27aee98548fc43e5bdfc329dd2adf33c/roles/4559b9a0837249f895a13c70096f8d38 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:21:45.352794 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8287ff0d-54db-47cd-9088-735fa52e6238 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:45.353354 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8287ff0d-54db-47cd-9088-735fa52e6238 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:45.353603 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8287ff0d-54db-47cd-9088-735fa52e6238 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:45.354962 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 537/1085] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:21:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:21:45.360105 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9b712d-26ed-4d14-b5f6-5aa326e81f60 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:45.360506 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9b712d-26ed-4d14-b5f6-5aa326e81f60 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:45.360653 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9b712d-26ed-4d14-b5f6-5aa326e81f60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:45.379965 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-0e9b712d-26ed-4d14-b5f6-5aa326e81f60 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:21:45.381184 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-0e9b712d-26ed-4d14-b5f6-5aa326e81f60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:21:45.449354 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 549/1086] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:21:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:21:45.472184 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8287ff0d-54db-47cd-9088-735fa52e6238 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:45.573718 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8287ff0d-54db-47cd-9088-735fa52e6238 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:45.585915 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:45.607391 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fa8f07-19b9-4a56-a2da-a970a2fe020c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:45.607958 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fa8f07-19b9-4a56-a2da-a970a2fe020c None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:45.608238 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fa8f07-19b9-4a56-a2da-a970a2fe020c None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:45.608514 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fa8f07-19b9-4a56-a2da-a970a2fe020c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:45.619372 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fa8f07-19b9-4a56-a2da-a970a2fe020c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:21:45.624206 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-50fa8f07-19b9-4a56-a2da-a970a2fe020c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 06 01:21:45.624206 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:45.624206 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:21:45.624206 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:21:45.624206 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:21:45.624206 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:21:45.624206 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:21:45.624206 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:21:45.624206 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:21:45.624206 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:21:45.624206 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:21:45.624206 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:21:45.624206 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:21:45.624206 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:21:45.624206 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:21:45.624206 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:21:45.625158 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:21:45.625158 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:21:45.625158 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:21:45.625158 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:21:45.625158 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:21:45.625158 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:21:45.625158 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:21:45.625158 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:21:45.625158 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:21:45.625158 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:21:45.625158 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:21:45.625158 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:21:45.625158 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:21:45.625158 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:21:45.625158 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:21:45.625158 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:21:45.626063 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:21:45.626063 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:21:45.626063 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:21:45.626063 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:21:45.626063 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:21:45.626063 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:21:45.626063 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:21:45.626063 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:21:45.626063 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:21:45.626063 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:21:45.626063 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 06 01:21:45.626063 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:21:45.626063 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 538/1087] 10.222.0.26 () {56 vars in 1120 bytes} [Sat Aug 6 01:21:45 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:21:45.631326 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9b712d-26ed-4d14-b5f6-5aa326e81f60 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:45.725192 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:45.729842 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9b712d-26ed-4d14-b5f6-5aa326e81f60 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:45.764922 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7c6a64-497c-4015-9d4e-846034db0c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:45.765557 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7c6a64-497c-4015-9d4e-846034db0c59 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:45.765839 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7c6a64-497c-4015-9d4e-846034db0c59 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:45.766113 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7c6a64-497c-4015-9d4e-846034db0c59 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:45.766825 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7c6a64-497c-4015-9d4e-846034db0c59 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:45.767135 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7c6a64-497c-4015-9d4e-846034db0c59 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:45.768488 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7c6a64-497c-4015-9d4e-846034db0c59 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:45.776235 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 550/1088] 10.222.0.26 () {56 vars in 1115 bytes} [Sat Aug 6 01:21:45 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:45.781795 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fa8f07-19b9-4a56-a2da-a970a2fe020c None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:45.881357 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fa8f07-19b9-4a56-a2da-a970a2fe020c None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:45.914501 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb04a5e1-b819-4d13-84bc-10c56c256f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:45.915141 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb04a5e1-b819-4d13-84bc-10c56c256f17 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:45.915934 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb04a5e1-b819-4d13-84bc-10c56c256f17 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:45.916210 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb04a5e1-b819-4d13-84bc-10c56c256f17 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:45.926515 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb04a5e1-b819-4d13-84bc-10c56c256f17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:45.927956 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb04a5e1-b819-4d13-84bc-10c56c256f17 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:45.942413 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-fb04a5e1-b819-4d13-84bc-10c56c256f17 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:21:45.942413 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:45.942413 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:21:45.942413 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:21:45.942413 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:21:45.942413 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:21:45.942413 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:21:45.942413 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:21:45.942413 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:21:45.942413 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:21:45.942413 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:21:45.942413 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:21:45.942413 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:21:45.942413 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:21:45.942413 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:21:45.943458 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:21:45.943458 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:21:45.943458 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:21:45.943458 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:21:45.943458 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:21:45.943458 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:21:45.943458 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:21:45.943458 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:21:45.943458 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:21:45.943458 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:21:45.943458 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:21:45.943458 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:21:45.943458 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:21:45.943458 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:21:45.943458 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:21:45.943458 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:21:45.943458 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:21:45.944340 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:21:45.944340 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:21:45.944340 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:21:45.944340 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:21:45.944340 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 539/1089] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:21:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:21:45.949999 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7c6a64-497c-4015-9d4e-846034db0c59 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:46.048545 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7c6a64-497c-4015-9d4e-846034db0c59 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:46.082829 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59b8be4-a079-46eb-b3e9-5bb6df6a3e3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:46.083445 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59b8be4-a079-46eb-b3e9-5bb6df6a3e3d None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:46.083725 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59b8be4-a079-46eb-b3e9-5bb6df6a3e3d None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:46.083996 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59b8be4-a079-46eb-b3e9-5bb6df6a3e3d None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:46.084681 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59b8be4-a079-46eb-b3e9-5bb6df6a3e3d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:46.084981 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59b8be4-a079-46eb-b3e9-5bb6df6a3e3d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:46.086362 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59b8be4-a079-46eb-b3e9-5bb6df6a3e3d None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:46.103472 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 551/1090] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Aug 6 01:21:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:46.104662 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb04a5e1-b819-4d13-84bc-10c56c256f17 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:46.130289 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:46.205429 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb04a5e1-b819-4d13-84bc-10c56c256f17 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:46.238129 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43507f6-e41c-4b4b-96eb-4110168fd809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:46.238762 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43507f6-e41c-4b4b-96eb-4110168fd809 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:46.239046 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43507f6-e41c-4b4b-96eb-4110168fd809 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:46.239326 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43507f6-e41c-4b4b-96eb-4110168fd809 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:46.247132 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43507f6-e41c-4b4b-96eb-4110168fd809 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:46.248574 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43507f6-e41c-4b4b-96eb-4110168fd809 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:46.260238 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-a43507f6-e41c-4b4b-96eb-4110168fd809 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 06 01:21:46.260238 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:46.260238 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:21:46.260238 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:21:46.260238 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:21:46.260238 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:21:46.260238 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:21:46.260238 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:21:46.260238 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:21:46.260238 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:21:46.260238 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:21:46.260238 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:21:46.260238 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 01:21:46.260238 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 01:21:46.260238 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 01:21:46.260238 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 01:21:46.261210 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:21:46.261210 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:21:46.261210 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 01:21:46.261210 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:21:46.261210 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 01:21:46.261210 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:21:46.261210 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:21:46.261210 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:21:46.261210 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:21:46.261210 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:21:46.261210 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:21:46.261210 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:21:46.261210 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:21:46.261210 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:21:46.261210 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:21:46.261210 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:21:46.261210 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:21:46.261210 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:21:46.262152 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:21:46.262152 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:21:46.262152 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 01:21:46.262152 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 01:21:46.262152 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 01:21:46.262152 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 01:21:46.262152 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 01:21:46.262152 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 01:21:46.262152 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 06 01:21:46.262152 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:21:46.262152 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 540/1091] 10.222.0.26 () {56 vars in 1093 bytes} [Sat Aug 6 01:21:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:21:46.267101 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59b8be4-a079-46eb-b3e9-5bb6df6a3e3d None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:46.288972 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:46.367287 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59b8be4-a079-46eb-b3e9-5bb6df6a3e3d None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:46.404111 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597e475f-99e9-4028-add7-c614a5a3de56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:46.408012 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597e475f-99e9-4028-add7-c614a5a3de56 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:46.408012 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597e475f-99e9-4028-add7-c614a5a3de56 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:46.408012 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597e475f-99e9-4028-add7-c614a5a3de56 None admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:46.408012 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597e475f-99e9-4028-add7-c614a5a3de56 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:46.408335 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d69c0ad-4163-40d0-bb6b-9d1625189007 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:46.408335 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d69c0ad-4163-40d0-bb6b-9d1625189007 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:46.408789 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597e475f-99e9-4028-add7-c614a5a3de56 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:46.408861 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d69c0ad-4163-40d0-bb6b-9d1625189007 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:46.408943 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597e475f-99e9-4028-add7-c614a5a3de56 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:46.409014 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 541/1092] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:21:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 01:21:46.413098 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3074d39e-0bf1-4a00-8748-89715e4a09af None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:46.413390 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3074d39e-0bf1-4a00-8748-89715e4a09af None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:46.413671 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3074d39e-0bf1-4a00-8748-89715e4a09af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:46.425553 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 552/1093] 10.222.0.26 () {56 vars in 1133 bytes} [Sat Aug 6 01:21:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:46.431252 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597e475f-99e9-4028-add7-c614a5a3de56 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:46.431961 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-3074d39e-0bf1-4a00-8748-89715e4a09af None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:21:46.433212 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-3074d39e-0bf1-4a00-8748-89715e4a09af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:21:46.500433 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 542/1094] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:21:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 06 01:21:46.508032 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:46.523997 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3074d39e-0bf1-4a00-8748-89715e4a09af None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:46.530218 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597e475f-99e9-4028-add7-c614a5a3de56 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:46.539195 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:46.563712 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a32710e-df36-4a36-ab3f-d9b75c5048e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:46.564321 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a32710e-df36-4a36-ab3f-d9b75c5048e6 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:46.564604 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a32710e-df36-4a36-ab3f-d9b75c5048e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:46.564886 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a32710e-df36-4a36-ab3f-d9b75c5048e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:46.622988 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3074d39e-0bf1-4a00-8748-89715e4a09af None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:46.633828 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a32710e-df36-4a36-ab3f-d9b75c5048e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:46.635817 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a32710e-df36-4a36-ab3f-d9b75c5048e6 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:46.656808 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c83399d-427f-4ad0-9ca6-dedbc5acd795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:46.657478 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c83399d-427f-4ad0-9ca6-dedbc5acd795 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:46.657769 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c83399d-427f-4ad0-9ca6-dedbc5acd795 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:46.658045 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c83399d-427f-4ad0-9ca6-dedbc5acd795 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:46.668266 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c83399d-427f-4ad0-9ca6-dedbc5acd795 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:21:46.673213 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-0c83399d-427f-4ad0-9ca6-dedbc5acd795 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:21:46.673213 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:46.673213 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:21:46.673213 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:21:46.673213 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:21:46.673213 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:21:46.673213 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:21:46.673213 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:21:46.673213 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:21:46.673213 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:21:46.673213 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:21:46.673213 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:21:46.673213 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:21:46.673213 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:21:46.673213 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:21:46.673213 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:21:46.674119 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:21:46.674119 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:21:46.674119 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:21:46.674119 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:21:46.674119 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:21:46.674119 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:21:46.674119 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:21:46.674119 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:21:46.674119 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:21:46.674119 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:21:46.674119 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:21:46.674119 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:21:46.674119 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:21:46.674119 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:21:46.674119 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:21:46.674119 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:21:46.674989 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:21:46.674989 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:21:46.674989 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:21:46.674989 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:21:46.674989 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:21:46.674989 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:21:46.674989 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:21:46.674989 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:21:46.674989 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:21:46.674989 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:21:46.674989 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:21:46.674989 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:21:46.674989 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 543/1095] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Aug 6 01:21:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:21:46.680373 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c83399d-427f-4ad0-9ca6-dedbc5acd795 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:46.712425 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:46.736044 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 553/1096] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Aug 6 01:21:46 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 06 01:21:46.741772 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a32710e-df36-4a36-ab3f-d9b75c5048e6 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:46.754568 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:46.779905 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c83399d-427f-4ad0-9ca6-dedbc5acd795 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:46.814819 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d25692-2c4a-4eba-8141-685f30e2be79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:46.815436 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d25692-2c4a-4eba-8141-685f30e2be79 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:46.815721 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d25692-2c4a-4eba-8141-685f30e2be79 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:46.815996 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d25692-2c4a-4eba-8141-685f30e2be79 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:46.816709 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d25692-2c4a-4eba-8141-685f30e2be79 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:46.817017 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d25692-2c4a-4eba-8141-685f30e2be79 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:46.818438 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d25692-2c4a-4eba-8141-685f30e2be79 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:46.826784 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 544/1097] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Aug 6 01:21:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:21:46.832432 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d25692-2c4a-4eba-8141-685f30e2be79 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:46.840301 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a32710e-df36-4a36-ab3f-d9b75c5048e6 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:46.874425 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9ed505-554f-40ab-b9c7-9cc4fe1a8ed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:46.875053 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9ed505-554f-40ab-b9c7-9cc4fe1a8ed9 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:46.875328 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9ed505-554f-40ab-b9c7-9cc4fe1a8ed9 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:46.875600 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9ed505-554f-40ab-b9c7-9cc4fe1a8ed9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:46.885613 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9ed505-554f-40ab-b9c7-9cc4fe1a8ed9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:46.887098 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9ed505-554f-40ab-b9c7-9cc4fe1a8ed9 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:46.902129 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-3e9ed505-554f-40ab-b9c7-9cc4fe1a8ed9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:21:46.902129 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:46.902129 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:21:46.902129 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:21:46.902129 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:21:46.902129 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:21:46.902129 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:21:46.902129 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:21:46.902129 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:21:46.902129 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:21:46.902129 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:21:46.902129 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:21:46.902129 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:21:46.902129 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:21:46.902129 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:21:46.903211 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:21:46.903211 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:21:46.903211 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:21:46.903211 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:21:46.903211 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:21:46.903211 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:21:46.903211 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:21:46.903211 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:21:46.903211 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:21:46.903211 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:21:46.903211 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:21:46.903211 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:21:46.903211 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:21:46.903211 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:21:46.903211 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:21:46.903211 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:21:46.903211 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:21:46.904133 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:21:46.904133 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:21:46.904133 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:21:46.904133 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:21:46.904133 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 554/1098] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:21:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:21:46.908966 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9ed505-554f-40ab-b9c7-9cc4fe1a8ed9 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:46.934347 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d25692-2c4a-4eba-8141-685f30e2be79 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:46.938884 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:46.967585 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf20ee65-c5b6-4acf-b09a-2bc4d14fca65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:46.968217 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf20ee65-c5b6-4acf-b09a-2bc4d14fca65 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:46.968495 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf20ee65-c5b6-4acf-b09a-2bc4d14fca65 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:46.968775 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf20ee65-c5b6-4acf-b09a-2bc4d14fca65 None admin] PATH_INFO: `/v3/users/27aee98548fc43e5bdfc329dd2adf33c` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:46.969970 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf20ee65-c5b6-4acf-b09a-2bc4d14fca65 None admin] RBAC: Authorizing `identity:get_user(user_id=27aee98548fc43e5bdfc329dd2adf33c)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:46.971373 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf20ee65-c5b6-4acf-b09a-2bc4d14fca65 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:46.972970 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 545/1099] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Aug 6 01:21:46 2022] GET /identity/v3/users/27aee98548fc43e5bdfc329dd2adf33c => generated 285 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:21:46.978922 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf20ee65-c5b6-4acf-b09a-2bc4d14fca65 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:46.982583 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:47.013476 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9ed505-554f-40ab-b9c7-9cc4fe1a8ed9 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:47.053951 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c4bd93-937b-440d-be78-7edc3423109f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:47.054575 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c4bd93-937b-440d-be78-7edc3423109f None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:47.054856 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c4bd93-937b-440d-be78-7edc3423109f None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:47.055131 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c4bd93-937b-440d-be78-7edc3423109f None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:47.055819 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c4bd93-937b-440d-be78-7edc3423109f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:47.056661 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c4bd93-937b-440d-be78-7edc3423109f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:47.058056 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c4bd93-937b-440d-be78-7edc3423109f None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:47.069547 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 555/1100] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Aug 6 01:21:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:47.076919 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c4bd93-937b-440d-be78-7edc3423109f None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:47.085355 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf20ee65-c5b6-4acf-b09a-2bc4d14fca65 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:47.121960 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c53de26-900a-4794-b8a6-251949864d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:47.122581 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c53de26-900a-4794-b8a6-251949864d07 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:47.122863 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c53de26-900a-4794-b8a6-251949864d07 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:47.123138 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c53de26-900a-4794-b8a6-251949864d07 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:47.168331 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:47.179007 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c4bd93-937b-440d-be78-7edc3423109f None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:47.198893 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c53de26-900a-4794-b8a6-251949864d07 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:47.200355 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c53de26-900a-4794-b8a6-251949864d07 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:47.203285 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:47.218360 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5a325c-957b-4ccd-92e9-917041efc938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:47.219008 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5a325c-957b-4ccd-92e9-917041efc938 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:47.219290 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5a325c-957b-4ccd-92e9-917041efc938 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:47.219570 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5a325c-957b-4ccd-92e9-917041efc938 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:47.226904 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5a325c-957b-4ccd-92e9-917041efc938 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:47.228600 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5a325c-957b-4ccd-92e9-917041efc938 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:47.240362 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-4b5a325c-957b-4ccd-92e9-917041efc938 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:21:47.240362 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:21:47.240362 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:21:47.240362 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:21:47.240362 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:21:47.240362 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:21:47.240362 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:21:47.240362 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:21:47.240362 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:21:47.240362 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:21:47.240362 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:21:47.240362 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:21:47.240362 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 01:21:47.240362 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 01:21:47.240362 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 01:21:47.241416 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 01:21:47.241416 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:21:47.241416 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:21:47.241416 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:21:47.241416 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:21:47.241416 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:21:47.241416 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:21:47.241416 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:21:47.241416 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:21:47.241416 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:21:47.241416 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:21:47.241416 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:21:47.241416 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:21:47.241416 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:21:47.241416 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:21:47.241416 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:21:47.241416 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:21:47.242315 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 01:21:47.242315 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 01:21:47.242315 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 01:21:47.242315 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 01:21:47.242315 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 01:21:47.242315 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 01:21:47.242315 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:21:47.242315 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:21:47.242315 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 556/1101] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Aug 6 01:21:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:21:47.247717 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5a325c-957b-4ccd-92e9-917041efc938 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:47.328484 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 546/1102] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Aug 6 01:21:46 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 06 01:21:47.329965 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c53de26-900a-4794-b8a6-251949864d07 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:47.356754 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5a325c-957b-4ccd-92e9-917041efc938 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:47.397373 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01210c77-138d-4909-843d-006ba026aba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:47.398011 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01210c77-138d-4909-843d-006ba026aba0 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:47.398298 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01210c77-138d-4909-843d-006ba026aba0 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:47.398578 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01210c77-138d-4909-843d-006ba026aba0 None admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:47.399335 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01210c77-138d-4909-843d-006ba026aba0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:47.399649 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01210c77-138d-4909-843d-006ba026aba0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:47.404428 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01210c77-138d-4909-843d-006ba026aba0 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:47.415277 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 557/1103] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Aug 6 01:21:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:21:47.422051 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01210c77-138d-4909-843d-006ba026aba0 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:47.432362 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c53de26-900a-4794-b8a6-251949864d07 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:47.435314 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:47.435577 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:47.470219 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c12337d-caa3-497b-a768-d83c60b05f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:47.470862 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c12337d-caa3-497b-a768-d83c60b05f00 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:47.471147 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c12337d-caa3-497b-a768-d83c60b05f00 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:47.471425 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c12337d-caa3-497b-a768-d83c60b05f00 None admin] PATH_INFO: `/v3/projects/36b6ddf715a747079ae3b654c838f46f` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:47.472609 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c12337d-caa3-497b-a768-d83c60b05f00 None admin] RBAC: Authorizing `identity:get_project(project_id=36b6ddf715a747079ae3b654c838f46f)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:47.474082 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c12337d-caa3-497b-a768-d83c60b05f00 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:47.475774 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 547/1104] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Aug 6 01:21:47 2022] GET /identity/v3/projects/36b6ddf715a747079ae3b654c838f46f => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:21:47.483060 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c12337d-caa3-497b-a768-d83c60b05f00 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:47.534557 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01210c77-138d-4909-843d-006ba026aba0 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:47.575539 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d60d496-1881-4f07-8398-ee903bf0c77d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:47.576443 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d60d496-1881-4f07-8398-ee903bf0c77d None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:47.576768 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d60d496-1881-4f07-8398-ee903bf0c77d None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:47.577042 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d60d496-1881-4f07-8398-ee903bf0c77d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:47.578114 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d60d496-1881-4f07-8398-ee903bf0c77d None admin] RBAC: Adding query filter params (role.id=f42c599286be4e0fa632dc6769f95753, scope.project.id=d9d68d310bf24449a5fe20d2dc264a58, user.id=b87adc1d5bc240e99f95f6595b350be4) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:47.578432 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d60d496-1881-4f07-8398-ee903bf0c77d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:47.580688 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d60d496-1881-4f07-8398-ee903bf0c77d None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:47.584454 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c12337d-caa3-497b-a768-d83c60b05f00 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:47.599996 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 558/1105] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Aug 6 01:21:47 2022] GET /identity/v3/role_assignments?user.id=b87adc1d5bc240e99f95f6595b350be4&scope.project.id=d9d68d310bf24449a5fe20d2dc264a58&role.id=f42c599286be4e0fa632dc6769f95753 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:21:47.624521 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ab7a7c-e959-40fd-9d82-bdcb4e8fb74e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:47.625608 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ab7a7c-e959-40fd-9d82-bdcb4e8fb74e None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:47.625608 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ab7a7c-e959-40fd-9d82-bdcb4e8fb74e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:47.625800 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ab7a7c-e959-40fd-9d82-bdcb4e8fb74e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:47.626528 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ab7a7c-e959-40fd-9d82-bdcb4e8fb74e None admin] RBAC: Adding query filter params (role.id=4559b9a0837249f895a13c70096f8d38, scope.project.id=36b6ddf715a747079ae3b654c838f46f, user.id=27aee98548fc43e5bdfc329dd2adf33c) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:47.626839 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ab7a7c-e959-40fd-9d82-bdcb4e8fb74e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:47.628898 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ab7a7c-e959-40fd-9d82-bdcb4e8fb74e None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:47.636033 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 548/1106] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Aug 6 01:21:47 2022] GET /identity/v3/role_assignments?user.id=27aee98548fc43e5bdfc329dd2adf33c&scope.project.id=36b6ddf715a747079ae3b654c838f46f&role.id=4559b9a0837249f895a13c70096f8d38 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:21:49.713442 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c6b35d-2a17-41e5-b91d-4872bec6bf55 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:49.713952 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c6b35d-2a17-41e5-b91d-4872bec6bf55 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:49.714044 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c6b35d-2a17-41e5-b91d-4872bec6bf55 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:49.715321 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 559/1107] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:21:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:21:49.720389 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286231c9-4e21-41c1-9b5f-88daa40b7877 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:49.720812 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286231c9-4e21-41c1-9b5f-88daa40b7877 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:49.720980 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286231c9-4e21-41c1-9b5f-88daa40b7877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:49.740010 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-286231c9-4e21-41c1-9b5f-88daa40b7877 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:21:49.741247 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-286231c9-4e21-41c1-9b5f-88daa40b7877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:21:49.769905 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:49.807470 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 549/1108] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:21:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:21:49.832023 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c6b35d-2a17-41e5-b91d-4872bec6bf55 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:49.905328 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:49.930544 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c6b35d-2a17-41e5-b91d-4872bec6bf55 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:49.963229 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fb7ddb-f8c1-4997-ab76-a9ecf22f08fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:49.963896 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fb7ddb-f8c1-4997-ab76-a9ecf22f08fe None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:49.964124 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fb7ddb-f8c1-4997-ab76-a9ecf22f08fe None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:49.964393 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fb7ddb-f8c1-4997-ab76-a9ecf22f08fe None admin] PATH_INFO: `/v3/domains/default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:49.965555 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fb7ddb-f8c1-4997-ab76-a9ecf22f08fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:49.966969 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fb7ddb-f8c1-4997-ab76-a9ecf22f08fe None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:49.968509 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 560/1109] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:21:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:49.974185 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286231c9-4e21-41c1-9b5f-88daa40b7877 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:50.069475 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286231c9-4e21-41c1-9b5f-88daa40b7877 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:50.102589 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b99849c-2cef-437d-9925-62c0db8a3ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:50.103829 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b99849c-2cef-437d-9925-62c0db8a3ed2 None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:50.104106 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b99849c-2cef-437d-9925-62c0db8a3ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:50.104379 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b99849c-2cef-437d-9925-62c0db8a3ed2 None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:50.105088 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b99849c-2cef-437d-9925-62c0db8a3ed2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:50.106510 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b99849c-2cef-437d-9925-62c0db8a3ed2 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:50.127159 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 550/1110] 10.222.0.26 () {60 vars in 1132 bytes} [Sat Aug 6 01:21:49 2022] POST /identity/v3/users => generated 286 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:21:52.144867 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f857eca9-164c-46d2-9bd4-b63504873682 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:52.145367 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f857eca9-164c-46d2-9bd4-b63504873682 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:52.145486 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f857eca9-164c-46d2-9bd4-b63504873682 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:52.146830 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 561/1111] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:21:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:21:52.152446 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ca5cdd-8064-4bf2-9421-c2556b775e93 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:52.152877 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ca5cdd-8064-4bf2-9421-c2556b775e93 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:52.153001 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ca5cdd-8064-4bf2-9421-c2556b775e93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:52.171331 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-43ca5cdd-8064-4bf2-9421-c2556b775e93 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:21:52.172510 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-43ca5cdd-8064-4bf2-9421-c2556b775e93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:21:52.193138 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:52.244527 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 551/1112] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:21:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:21:52.267631 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f857eca9-164c-46d2-9bd4-b63504873682 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:52.366119 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f857eca9-164c-46d2-9bd4-b63504873682 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:52.398702 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c20c7c-b2c0-49c7-a66a-34b8a4d6fea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:52.399335 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c20c7c-b2c0-49c7-a66a-34b8a4d6fea4 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:52.400241 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c20c7c-b2c0-49c7-a66a-34b8a4d6fea4 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:52.400513 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c20c7c-b2c0-49c7-a66a-34b8a4d6fea4 None admin] PATH_INFO: `/v3/roles/6d8b63d350944d78812edbefb32ae8ee` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:52.401974 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c20c7c-b2c0-49c7-a66a-34b8a4d6fea4 None admin] RBAC: Authorizing `identity:get_role(role_id=6d8b63d350944d78812edbefb32ae8ee)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:52.403325 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c20c7c-b2c0-49c7-a66a-34b8a4d6fea4 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:52.404647 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 562/1113] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Aug 6 01:21:52 2022] GET /identity/v3/roles/6d8b63d350944d78812edbefb32ae8ee => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:52.410565 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ca5cdd-8064-4bf2-9421-c2556b775e93 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:52.506917 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ca5cdd-8064-4bf2-9421-c2556b775e93 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:52.539635 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a9ebca-f6ef-4a54-a7d1-e9fc0323ba27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:52.540896 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a9ebca-f6ef-4a54-a7d1-e9fc0323ba27 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:52.541392 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a9ebca-f6ef-4a54-a7d1-e9fc0323ba27 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:52.541747 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a9ebca-f6ef-4a54-a7d1-e9fc0323ba27 None admin] PATH_INFO: `/v3/users/6a10540ed55f4a3c9fc03185e2837fdd` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:52.554594 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a9ebca-f6ef-4a54-a7d1-e9fc0323ba27 None admin] RBAC: Authorizing `identity:get_user(user_id=6a10540ed55f4a3c9fc03185e2837fdd)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:52.556692 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a9ebca-f6ef-4a54-a7d1-e9fc0323ba27 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:52.558358 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 552/1114] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Aug 6 01:21:52 2022] GET /identity/v3/users/6a10540ed55f4a3c9fc03185e2837fdd => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:52.564132 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c20c7c-b2c0-49c7-a66a-34b8a4d6fea4 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:52.576641 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:52.661237 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c20c7c-b2c0-49c7-a66a-34b8a4d6fea4 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:52.693063 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a389d8-0f6c-4b74-999d-34b85104d226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:52.693689 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a389d8-0f6c-4b74-999d-34b85104d226 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:52.693972 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a389d8-0f6c-4b74-999d-34b85104d226 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:52.694249 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a389d8-0f6c-4b74-999d-34b85104d226 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:52.760209 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a389d8-0f6c-4b74-999d-34b85104d226 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:52.761678 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a389d8-0f6c-4b74-999d-34b85104d226 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:52.772056 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:52.855902 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 563/1115] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Aug 6 01:21:52 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 06 01:21:52.861809 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a9ebca-f6ef-4a54-a7d1-e9fc0323ba27 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:52.874397 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:52.959626 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a9ebca-f6ef-4a54-a7d1-e9fc0323ba27 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:52.992187 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241e172a-688f-42b6-8626-c1980de89f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:52.992824 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241e172a-688f-42b6-8626-c1980de89f72 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:52.993134 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241e172a-688f-42b6-8626-c1980de89f72 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:52.993973 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241e172a-688f-42b6-8626-c1980de89f72 None admin] PATH_INFO: `/v3/projects/36b6ddf715a747079ae3b654c838f46f` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:52.995131 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241e172a-688f-42b6-8626-c1980de89f72 None admin] RBAC: Authorizing `identity:get_project(project_id=36b6ddf715a747079ae3b654c838f46f)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:52.996553 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241e172a-688f-42b6-8626-c1980de89f72 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:52.998271 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 553/1116] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Aug 6 01:21:52 2022] GET /identity/v3/projects/36b6ddf715a747079ae3b654c838f46f => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:53.004349 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a389d8-0f6c-4b74-999d-34b85104d226 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:53.107421 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a389d8-0f6c-4b74-999d-34b85104d226 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:53.119309 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:53.142192 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386a76fa-a1e5-4a30-ad74-87e29cc3c837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:53.142802 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386a76fa-a1e5-4a30-ad74-87e29cc3c837 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:53.143085 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386a76fa-a1e5-4a30-ad74-87e29cc3c837 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:53.143359 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386a76fa-a1e5-4a30-ad74-87e29cc3c837 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:53.144150 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386a76fa-a1e5-4a30-ad74-87e29cc3c837 None admin] RBAC: Adding query filter params (role.id=6d8b63d350944d78812edbefb32ae8ee, scope.project.id=36b6ddf715a747079ae3b654c838f46f, user.id=6a10540ed55f4a3c9fc03185e2837fdd) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:53.144464 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386a76fa-a1e5-4a30-ad74-87e29cc3c837 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:53.145918 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386a76fa-a1e5-4a30-ad74-87e29cc3c837 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:53.151394 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 564/1117] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Aug 6 01:21:53 2022] GET /identity/v3/role_assignments?user.id=6a10540ed55f4a3c9fc03185e2837fdd&scope.project.id=36b6ddf715a747079ae3b654c838f46f&role.id=6d8b63d350944d78812edbefb32ae8ee => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:55.127720 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a8f3b6-f6cc-4a93-bde9-22eacd3f8ff5 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:55.128299 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a8f3b6-f6cc-4a93-bde9-22eacd3f8ff5 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:55.128299 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a8f3b6-f6cc-4a93-bde9-22eacd3f8ff5 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:55.129649 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 554/1118] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:21:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:21:55.134803 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d88946-2239-4b95-a2a2-c2d3bf4b0cea None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:55.135247 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d88946-2239-4b95-a2a2-c2d3bf4b0cea None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:55.135348 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d88946-2239-4b95-a2a2-c2d3bf4b0cea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:55.153558 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-d8d88946-2239-4b95-a2a2-c2d3bf4b0cea None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:21:55.154734 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-d8d88946-2239-4b95-a2a2-c2d3bf4b0cea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:21:55.218915 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 565/1119] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:21:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:21:55.241670 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a8f3b6-f6cc-4a93-bde9-22eacd3f8ff5 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:55.317756 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:55.340170 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a8f3b6-f6cc-4a93-bde9-22eacd3f8ff5 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:55.374014 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44cf9fa-5a99-4106-9743-7a4b5091f64b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:55.374642 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44cf9fa-5a99-4106-9743-7a4b5091f64b None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:55.374924 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44cf9fa-5a99-4106-9743-7a4b5091f64b None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:55.375201 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44cf9fa-5a99-4106-9743-7a4b5091f64b None admin] PATH_INFO: `/v3/roles/6d8b63d350944d78812edbefb32ae8ee` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:55.376665 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44cf9fa-5a99-4106-9743-7a4b5091f64b None admin] RBAC: Authorizing `identity:get_role(role_id=6d8b63d350944d78812edbefb32ae8ee)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:55.378112 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44cf9fa-5a99-4106-9743-7a4b5091f64b None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:55.379381 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 555/1120] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Aug 6 01:21:55 2022] GET /identity/v3/roles/6d8b63d350944d78812edbefb32ae8ee => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:55.385697 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d88946-2239-4b95-a2a2-c2d3bf4b0cea None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:55.480662 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d88946-2239-4b95-a2a2-c2d3bf4b0cea None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:55.482931 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:55.513636 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66237fca-c9e2-4253-b8dd-3d35faf1a4f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:55.514264 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66237fca-c9e2-4253-b8dd-3d35faf1a4f3 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:55.514544 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66237fca-c9e2-4253-b8dd-3d35faf1a4f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:55.514816 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66237fca-c9e2-4253-b8dd-3d35faf1a4f3 None admin] PATH_INFO: `/v3/users/6a10540ed55f4a3c9fc03185e2837fdd` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:55.515960 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66237fca-c9e2-4253-b8dd-3d35faf1a4f3 None admin] RBAC: Authorizing `identity:get_user(user_id=6a10540ed55f4a3c9fc03185e2837fdd)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:55.517406 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66237fca-c9e2-4253-b8dd-3d35faf1a4f3 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:55.518990 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 566/1121] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Aug 6 01:21:55 2022] GET /identity/v3/users/6a10540ed55f4a3c9fc03185e2837fdd => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:21:55.525416 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44cf9fa-5a99-4106-9743-7a4b5091f64b None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:55.620929 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44cf9fa-5a99-4106-9743-7a4b5091f64b None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:55.653762 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f7a926-79f1-48d5-afa4-810449fb4491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:55.654328 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f7a926-79f1-48d5-afa4-810449fb4491 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:55.654609 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f7a926-79f1-48d5-afa4-810449fb4491 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:55.654884 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f7a926-79f1-48d5-afa4-810449fb4491 None admin] PATH_INFO: `/v3/projects/36b6ddf715a747079ae3b654c838f46f` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:55.656029 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f7a926-79f1-48d5-afa4-810449fb4491 None admin] RBAC: Authorizing `identity:get_project(project_id=36b6ddf715a747079ae3b654c838f46f)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:55.657484 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f7a926-79f1-48d5-afa4-810449fb4491 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:55.659117 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 556/1122] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Aug 6 01:21:55 2022] GET /identity/v3/projects/36b6ddf715a747079ae3b654c838f46f => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:55.665596 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66237fca-c9e2-4253-b8dd-3d35faf1a4f3 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:55.764092 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66237fca-c9e2-4253-b8dd-3d35faf1a4f3 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:55.797583 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9689d037-0efd-4312-8bf6-65071c6668db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:55.798234 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9689d037-0efd-4312-8bf6-65071c6668db None admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:55.798513 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9689d037-0efd-4312-8bf6-65071c6668db None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:55.798787 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9689d037-0efd-4312-8bf6-65071c6668db None admin] PATH_INFO: `/v3/projects/36b6ddf715a747079ae3b654c838f46f/users/6a10540ed55f4a3c9fc03185e2837fdd/roles/6d8b63d350944d78812edbefb32ae8ee` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:55.800915 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9689d037-0efd-4312-8bf6-65071c6668db None admin] RBAC: Authorizing `identity:create_grant(project_id=36b6ddf715a747079ae3b654c838f46f, user_id=6a10540ed55f4a3c9fc03185e2837fdd, role_id=6d8b63d350944d78812edbefb32ae8ee)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:55.802353 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9689d037-0efd-4312-8bf6-65071c6668db None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:55.816098 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 567/1123] 10.222.0.26 () {58 vars in 1439 bytes} [Sat Aug 6 01:21:55 2022] PUT /identity/v3/projects/36b6ddf715a747079ae3b654c838f46f/users/6a10540ed55f4a3c9fc03185e2837fdd/roles/6d8b63d350944d78812edbefb32ae8ee => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:21:57.793414 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6877e0ed-834f-4be3-86dc-acb70e5ff616 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:57.793960 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6877e0ed-834f-4be3-86dc-acb70e5ff616 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:57.793960 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6877e0ed-834f-4be3-86dc-acb70e5ff616 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:57.795324 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 557/1124] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:21:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:21:57.800342 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23456b59-8548-4fdd-b376-2f3435181bdb None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:57.800715 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23456b59-8548-4fdd-b376-2f3435181bdb None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:57.800890 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23456b59-8548-4fdd-b376-2f3435181bdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:57.818464 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-23456b59-8548-4fdd-b376-2f3435181bdb None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:21:57.819631 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-23456b59-8548-4fdd-b376-2f3435181bdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:21:57.845184 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:57.885678 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 568/1125] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:21:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:21:57.908967 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6877e0ed-834f-4be3-86dc-acb70e5ff616 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:57.924459 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:58.007949 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6877e0ed-834f-4be3-86dc-acb70e5ff616 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:58.041008 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcba0d2d-3e4e-4be2-8c1b-585e1d23aa78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:58.041656 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcba0d2d-3e4e-4be2-8c1b-585e1d23aa78 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:58.041937 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcba0d2d-3e4e-4be2-8c1b-585e1d23aa78 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:58.042213 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcba0d2d-3e4e-4be2-8c1b-585e1d23aa78 None admin] PATH_INFO: `/v3/roles/6d8b63d350944d78812edbefb32ae8ee` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:58.043700 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcba0d2d-3e4e-4be2-8c1b-585e1d23aa78 None admin] RBAC: Authorizing `identity:get_role(role_id=6d8b63d350944d78812edbefb32ae8ee)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:58.045000 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcba0d2d-3e4e-4be2-8c1b-585e1d23aa78 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:58.046346 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 558/1126] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Aug 6 01:21:57 2022] GET /identity/v3/roles/6d8b63d350944d78812edbefb32ae8ee => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:58.052114 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23456b59-8548-4fdd-b376-2f3435181bdb None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:58.147661 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23456b59-8548-4fdd-b376-2f3435181bdb None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:58.181301 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ec9224-9762-4e10-a005-cfe9654e93a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:58.181935 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ec9224-9762-4e10-a005-cfe9654e93a5 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:58.182219 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ec9224-9762-4e10-a005-cfe9654e93a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:58.182497 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ec9224-9762-4e10-a005-cfe9654e93a5 None admin] PATH_INFO: `/v3/users/6a10540ed55f4a3c9fc03185e2837fdd` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:58.183653 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ec9224-9762-4e10-a005-cfe9654e93a5 None admin] RBAC: Authorizing `identity:get_user(user_id=6a10540ed55f4a3c9fc03185e2837fdd)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:58.185098 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ec9224-9762-4e10-a005-cfe9654e93a5 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:58.186669 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 569/1127] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Aug 6 01:21:58 2022] GET /identity/v3/users/6a10540ed55f4a3c9fc03185e2837fdd => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:58.192555 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcba0d2d-3e4e-4be2-8c1b-585e1d23aa78 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:58.272564 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:58.290356 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcba0d2d-3e4e-4be2-8c1b-585e1d23aa78 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:58.323954 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8812b62a-88d3-4a3b-bc25-77dad2e311c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:58.324505 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8812b62a-88d3-4a3b-bc25-77dad2e311c4 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:58.324792 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8812b62a-88d3-4a3b-bc25-77dad2e311c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:58.325094 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8812b62a-88d3-4a3b-bc25-77dad2e311c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:58.389748 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8812b62a-88d3-4a3b-bc25-77dad2e311c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:58.391203 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8812b62a-88d3-4a3b-bc25-77dad2e311c4 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:58.468366 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:58.487657 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 559/1128] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Aug 6 01:21:58 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 06 01:21:58.493924 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ec9224-9762-4e10-a005-cfe9654e93a5 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:58.523522 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:58.592407 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ec9224-9762-4e10-a005-cfe9654e93a5 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:58.625992 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61648b6d-a265-4085-95d8-53ae69d9eca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:58.626620 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61648b6d-a265-4085-95d8-53ae69d9eca4 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:58.626901 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61648b6d-a265-4085-95d8-53ae69d9eca4 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:58.627179 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61648b6d-a265-4085-95d8-53ae69d9eca4 None admin] PATH_INFO: `/v3/projects/36b6ddf715a747079ae3b654c838f46f` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:58.628313 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61648b6d-a265-4085-95d8-53ae69d9eca4 None admin] RBAC: Authorizing `identity:get_project(project_id=36b6ddf715a747079ae3b654c838f46f)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:58.629761 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61648b6d-a265-4085-95d8-53ae69d9eca4 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:58.631388 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 570/1129] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Aug 6 01:21:58 2022] GET /identity/v3/projects/36b6ddf715a747079ae3b654c838f46f => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:21:58.638165 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8812b62a-88d3-4a3b-bc25-77dad2e311c4 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:21:58.735621 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8812b62a-88d3-4a3b-bc25-77dad2e311c4 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:21:58.770077 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445fb56e-f5a2-4ea6-863d-93a89e0fdcd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:21:58.770682 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445fb56e-f5a2-4ea6-863d-93a89e0fdcd0 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:21:58.771014 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445fb56e-f5a2-4ea6-863d-93a89e0fdcd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:21:58.771366 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445fb56e-f5a2-4ea6-863d-93a89e0fdcd0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:21:58.772144 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445fb56e-f5a2-4ea6-863d-93a89e0fdcd0 None admin] RBAC: Adding query filter params (role.id=6d8b63d350944d78812edbefb32ae8ee, scope.project.id=36b6ddf715a747079ae3b654c838f46f, user.id=6a10540ed55f4a3c9fc03185e2837fdd) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:21:58.772455 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445fb56e-f5a2-4ea6-863d-93a89e0fdcd0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:21:58.773888 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445fb56e-f5a2-4ea6-863d-93a89e0fdcd0 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:21:58.780002 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 560/1130] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Aug 6 01:21:58 2022] GET /identity/v3/role_assignments?user.id=6a10540ed55f4a3c9fc03185e2837fdd&scope.project.id=36b6ddf715a747079ae3b654c838f46f&role.id=6d8b63d350944d78812edbefb32ae8ee => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:22:00.775387 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99413d1e-cc6b-4566-9a83-3a45eaf50285 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:00.775914 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99413d1e-cc6b-4566-9a83-3a45eaf50285 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:00.776027 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99413d1e-cc6b-4566-9a83-3a45eaf50285 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:00.777304 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 571/1131] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:22:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:22:00.782467 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ff06c4-50ba-48b9-a5ab-14eee20c3160 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:00.782851 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ff06c4-50ba-48b9-a5ab-14eee20c3160 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:00.783025 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ff06c4-50ba-48b9-a5ab-14eee20c3160 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:00.801644 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-b5ff06c4-50ba-48b9-a5ab-14eee20c3160 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:22:00.802825 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-b5ff06c4-50ba-48b9-a5ab-14eee20c3160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:00.845132 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:22:00.870029 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 561/1132] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:22:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:22:00.893403 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99413d1e-cc6b-4566-9a83-3a45eaf50285 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:00.988047 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:22:00.993508 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99413d1e-cc6b-4566-9a83-3a45eaf50285 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:01.029125 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc31c96-fc65-4e96-9d40-306ed93807db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:01.029697 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc31c96-fc65-4e96-9d40-306ed93807db None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:01.029979 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc31c96-fc65-4e96-9d40-306ed93807db None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:01.030247 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc31c96-fc65-4e96-9d40-306ed93807db None admin] PATH_INFO: `/v3/domains/default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:01.031378 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc31c96-fc65-4e96-9d40-306ed93807db None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:01.032785 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc31c96-fc65-4e96-9d40-306ed93807db None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:01.034378 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 572/1133] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:22:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:22:01.040141 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ff06c4-50ba-48b9-a5ab-14eee20c3160 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:01.135550 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ff06c4-50ba-48b9-a5ab-14eee20c3160 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:01.167389 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21effae5-af18-440a-8a02-8768f6e82f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:01.167991 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21effae5-af18-440a-8a02-8768f6e82f29 None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:01.168269 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21effae5-af18-440a-8a02-8768f6e82f29 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:01.168552 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21effae5-af18-440a-8a02-8768f6e82f29 None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:01.169244 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21effae5-af18-440a-8a02-8768f6e82f29 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:01.170671 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21effae5-af18-440a-8a02-8768f6e82f29 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:01.197371 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 562/1134] 10.222.0.26 () {60 vars in 1141 bytes} [Sat Aug 6 01:22:01 2022] POST /identity/v3/projects => generated 311 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:22:03.198172 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65daf8f-0469-42de-8546-21d7a61d4b8d None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:03.198813 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65daf8f-0469-42de-8546-21d7a61d4b8d None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:03.198813 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65daf8f-0469-42de-8546-21d7a61d4b8d None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:03.200046 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 573/1135] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:22:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:22:03.205336 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70538f94-c5b0-446b-8e4b-a991dac9971a None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:03.205710 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70538f94-c5b0-446b-8e4b-a991dac9971a None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:03.205904 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70538f94-c5b0-446b-8e4b-a991dac9971a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:03.224171 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-70538f94-c5b0-446b-8e4b-a991dac9971a None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:22:03.225399 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-70538f94-c5b0-446b-8e4b-a991dac9971a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:03.248375 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:22:03.293340 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 563/1136] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:22:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:22:03.317973 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65daf8f-0469-42de-8546-21d7a61d4b8d None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:03.418651 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65daf8f-0469-42de-8546-21d7a61d4b8d None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:03.451204 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bcc5be-cbd9-48a2-801a-492003509ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:03.451780 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bcc5be-cbd9-48a2-801a-492003509ed2 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:03.452063 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bcc5be-cbd9-48a2-801a-492003509ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:03.452341 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bcc5be-cbd9-48a2-801a-492003509ed2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:03.453538 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bcc5be-cbd9-48a2-801a-492003509ed2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:03.454973 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bcc5be-cbd9-48a2-801a-492003509ed2 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:03.456563 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 574/1137] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:22:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:22:03.463270 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70538f94-c5b0-446b-8e4b-a991dac9971a None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:03.558776 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70538f94-c5b0-446b-8e4b-a991dac9971a None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:03.591859 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b85634-2e76-4a26-b2f6-95c28cf2193c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:03.592468 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b85634-2e76-4a26-b2f6-95c28cf2193c None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:03.592754 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b85634-2e76-4a26-b2f6-95c28cf2193c None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:03.593036 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b85634-2e76-4a26-b2f6-95c28cf2193c None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:03.593748 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b85634-2e76-4a26-b2f6-95c28cf2193c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:03.595176 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b85634-2e76-4a26-b2f6-95c28cf2193c None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:03.616202 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 564/1138] 10.222.0.26 () {60 vars in 1132 bytes} [Sat Aug 6 01:22:03 2022] POST /identity/v3/users => generated 286 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:22:05.646527 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4344bc2-e575-450d-b85d-2794be7728a4 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:05.647043 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4344bc2-e575-450d-b85d-2794be7728a4 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:05.647043 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4344bc2-e575-450d-b85d-2794be7728a4 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:05.648334 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 575/1139] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:22:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:22:05.654023 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14be5f7-365b-4701-80a3-8e5cd1968329 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:05.654413 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14be5f7-365b-4701-80a3-8e5cd1968329 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:05.654575 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14be5f7-365b-4701-80a3-8e5cd1968329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:05.672091 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-f14be5f7-365b-4701-80a3-8e5cd1968329 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:22:05.673289 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-f14be5f7-365b-4701-80a3-8e5cd1968329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:05.682840 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:22:05.740499 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 565/1140] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:22:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:22:05.765246 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4344bc2-e575-450d-b85d-2794be7728a4 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:05.799276 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:22:05.865627 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4344bc2-e575-450d-b85d-2794be7728a4 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:05.900417 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92572b8-d8a9-4452-a8b5-41863b6f9d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:05.900975 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92572b8-d8a9-4452-a8b5-41863b6f9d14 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:05.901293 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92572b8-d8a9-4452-a8b5-41863b6f9d14 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:05.901568 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92572b8-d8a9-4452-a8b5-41863b6f9d14 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:05.911249 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92572b8-d8a9-4452-a8b5-41863b6f9d14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:22:05.917071 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-d92572b8-d8a9-4452-a8b5-41863b6f9d14 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:22:05.917071 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:22:05.917071 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:22:05.917071 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:22:05.917071 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:22:05.917071 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:22:05.917071 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:22:05.917071 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:22:05.917071 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:22:05.917071 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:22:05.917071 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:22:05.917071 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:22:05.917071 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:22:05.917071 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:22:05.917071 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:22:05.917071 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:22:05.918210 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:22:05.918210 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:22:05.918210 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:22:05.918210 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:22:05.918210 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:22:05.918210 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:22:05.918210 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:22:05.918210 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:22:05.918210 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:22:05.918210 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:22:05.918210 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:22:05.918210 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:22:05.918210 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:22:05.918210 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:22:05.918210 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:22:05.918210 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:22:05.919304 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:22:05.919304 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:22:05.919304 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:22:05.919304 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:22:05.919304 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:22:05.919304 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:22:05.919304 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:22:05.919304 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:22:05.919304 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:22:05.919304 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:22:05.919304 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:22:05.919304 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:22:05.919304 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 576/1141] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Aug 6 01:22:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:22:05.924195 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14be5f7-365b-4701-80a3-8e5cd1968329 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:06.020261 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14be5f7-365b-4701-80a3-8e5cd1968329 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:06.053123 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a75ea1-a19c-4e18-8959-b8683db66685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:06.053727 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a75ea1-a19c-4e18-8959-b8683db66685 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:06.054007 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a75ea1-a19c-4e18-8959-b8683db66685 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:06.054280 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a75ea1-a19c-4e18-8959-b8683db66685 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:06.054982 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a75ea1-a19c-4e18-8959-b8683db66685 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:22:06.055316 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a75ea1-a19c-4e18-8959-b8683db66685 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:06.057664 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a75ea1-a19c-4e18-8959-b8683db66685 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:06.065413 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 566/1142] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Aug 6 01:22:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:22:06.071605 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92572b8-d8a9-4452-a8b5-41863b6f9d14 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:06.162240 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:22:06.168953 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92572b8-d8a9-4452-a8b5-41863b6f9d14 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:06.201920 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4337af0b-89c6-4737-b941-4054366716b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:06.202540 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4337af0b-89c6-4737-b941-4054366716b1 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:06.202821 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4337af0b-89c6-4737-b941-4054366716b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:06.203093 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4337af0b-89c6-4737-b941-4054366716b1 None admin] PATH_INFO: `/v3/users/34d78e0cfdd3427d95df409f889cde2e` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:06.216270 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4337af0b-89c6-4737-b941-4054366716b1 None admin] RBAC: Authorizing `identity:get_user(user_id=34d78e0cfdd3427d95df409f889cde2e)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:06.217954 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4337af0b-89c6-4737-b941-4054366716b1 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:06.219575 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 577/1143] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Aug 6 01:22:06 2022] GET /identity/v3/users/34d78e0cfdd3427d95df409f889cde2e => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:22:06.225539 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a75ea1-a19c-4e18-8959-b8683db66685 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:06.235797 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:22:06.332419 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a75ea1-a19c-4e18-8959-b8683db66685 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:06.365074 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30572084-7571-4f0b-89e7-33c3ebd050c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:06.365700 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30572084-7571-4f0b-89e7-33c3ebd050c1 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:06.366014 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30572084-7571-4f0b-89e7-33c3ebd050c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:06.366321 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30572084-7571-4f0b-89e7-33c3ebd050c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:06.433559 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30572084-7571-4f0b-89e7-33c3ebd050c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:06.435012 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30572084-7571-4f0b-89e7-33c3ebd050c1 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:06.442513 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:22:06.530283 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 567/1144] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Aug 6 01:22:06 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 06 01:22:06.536182 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4337af0b-89c6-4737-b941-4054366716b1 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:06.634573 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4337af0b-89c6-4737-b941-4054366716b1 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:06.668531 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d767dd2-6e1d-41ff-8de7-7b655d27edab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:06.669188 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d767dd2-6e1d-41ff-8de7-7b655d27edab None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:06.669468 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d767dd2-6e1d-41ff-8de7-7b655d27edab None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:06.669739 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d767dd2-6e1d-41ff-8de7-7b655d27edab None admin] PATH_INFO: `/v3/projects/fedb3641c9d143aebb204db3ec4897f7` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:06.670879 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d767dd2-6e1d-41ff-8de7-7b655d27edab None admin] RBAC: Authorizing `identity:get_project(project_id=fedb3641c9d143aebb204db3ec4897f7)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:06.672290 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d767dd2-6e1d-41ff-8de7-7b655d27edab None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:06.673954 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 578/1145] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Aug 6 01:22:06 2022] GET /identity/v3/projects/fedb3641c9d143aebb204db3ec4897f7 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:22:06.680108 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30572084-7571-4f0b-89e7-33c3ebd050c1 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:06.778049 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30572084-7571-4f0b-89e7-33c3ebd050c1 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:06.784890 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:22:06.811156 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba25e62f-c008-45e4-b593-1d4bc9978536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:06.811782 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba25e62f-c008-45e4-b593-1d4bc9978536 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:06.812099 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba25e62f-c008-45e4-b593-1d4bc9978536 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:06.812409 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba25e62f-c008-45e4-b593-1d4bc9978536 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:06.813272 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba25e62f-c008-45e4-b593-1d4bc9978536 None admin] RBAC: Adding query filter params (role.id=4559b9a0837249f895a13c70096f8d38, scope.project.id=fedb3641c9d143aebb204db3ec4897f7, user.id=34d78e0cfdd3427d95df409f889cde2e) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:22:06.813603 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba25e62f-c008-45e4-b593-1d4bc9978536 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:06.815042 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba25e62f-c008-45e4-b593-1d4bc9978536 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:06.820345 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 568/1146] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Aug 6 01:22:06 2022] GET /identity/v3/role_assignments?user.id=34d78e0cfdd3427d95df409f889cde2e&scope.project.id=fedb3641c9d143aebb204db3ec4897f7&role.id=4559b9a0837249f895a13c70096f8d38 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:22:08.789966 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a52867-245f-4892-bca7-7b9726637d5d None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:08.790581 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a52867-245f-4892-bca7-7b9726637d5d None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:08.790698 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a52867-245f-4892-bca7-7b9726637d5d None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:08.792038 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 579/1147] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:22:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:22:08.797163 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1c4273-6ecf-404b-8599-b537cab15aa6 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:08.797577 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1c4273-6ecf-404b-8599-b537cab15aa6 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:08.797803 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1c4273-6ecf-404b-8599-b537cab15aa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:08.816370 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-cb1c4273-6ecf-404b-8599-b537cab15aa6 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:22:08.817612 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-cb1c4273-6ecf-404b-8599-b537cab15aa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:08.884946 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 569/1148] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:22:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:22:08.906754 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a52867-245f-4892-bca7-7b9726637d5d None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:08.928682 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:22:09.007098 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a52867-245f-4892-bca7-7b9726637d5d None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:09.039465 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f61905-b42d-4b36-97c3-647d05e89e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:09.040663 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f61905-b42d-4b36-97c3-647d05e89e1b None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:09.040894 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f61905-b42d-4b36-97c3-647d05e89e1b None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:09.041199 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f61905-b42d-4b36-97c3-647d05e89e1b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:09.050962 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f61905-b42d-4b36-97c3-647d05e89e1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:22:09.056567 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-74f61905-b42d-4b36-97c3-647d05e89e1b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:22:09.056567 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:22:09.056567 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:22:09.056567 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:22:09.056567 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:22:09.056567 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:22:09.056567 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:22:09.056567 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:22:09.056567 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:22:09.056567 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:22:09.056567 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:22:09.056567 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:22:09.056567 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:22:09.056567 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:22:09.056567 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:22:09.056567 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:22:09.057673 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:22:09.057673 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:22:09.057673 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:22:09.057673 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:22:09.057673 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:22:09.057673 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:22:09.057673 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:22:09.057673 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:22:09.057673 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:22:09.057673 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:22:09.057673 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:22:09.057673 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:22:09.057673 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:22:09.057673 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:22:09.057673 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:22:09.057673 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:22:09.058705 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:22:09.058705 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:22:09.058705 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:22:09.058705 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:22:09.058705 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:22:09.058705 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:22:09.058705 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:22:09.058705 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:22:09.058705 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:22:09.058705 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:22:09.058705 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:22:09.058705 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:22:09.058705 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 580/1149] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Aug 6 01:22:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:22:09.063683 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1c4273-6ecf-404b-8599-b537cab15aa6 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:09.158426 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:22:09.161599 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1c4273-6ecf-404b-8599-b537cab15aa6 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:09.195003 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb28e586-2e87-422e-9380-4f5e271c6c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:09.195615 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb28e586-2e87-422e-9380-4f5e271c6c02 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:09.196512 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb28e586-2e87-422e-9380-4f5e271c6c02 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:09.196809 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb28e586-2e87-422e-9380-4f5e271c6c02 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:09.197541 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb28e586-2e87-422e-9380-4f5e271c6c02 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:22:09.197850 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb28e586-2e87-422e-9380-4f5e271c6c02 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:09.199199 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb28e586-2e87-422e-9380-4f5e271c6c02 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:09.206967 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 570/1150] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Aug 6 01:22:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:22:09.213555 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f61905-b42d-4b36-97c3-647d05e89e1b None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:09.292606 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:22:09.311507 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f61905-b42d-4b36-97c3-647d05e89e1b None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:09.344392 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3393ec72-45ed-4847-b148-95284bf3aa1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:09.345014 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3393ec72-45ed-4847-b148-95284bf3aa1a None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:09.345323 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3393ec72-45ed-4847-b148-95284bf3aa1a None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:09.345596 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3393ec72-45ed-4847-b148-95284bf3aa1a None admin] PATH_INFO: `/v3/users/34d78e0cfdd3427d95df409f889cde2e` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:09.346741 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3393ec72-45ed-4847-b148-95284bf3aa1a None admin] RBAC: Authorizing `identity:get_user(user_id=34d78e0cfdd3427d95df409f889cde2e)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:09.348160 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3393ec72-45ed-4847-b148-95284bf3aa1a None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:09.349752 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 581/1151] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Aug 6 01:22:09 2022] GET /identity/v3/users/34d78e0cfdd3427d95df409f889cde2e => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:22:09.355870 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb28e586-2e87-422e-9380-4f5e271c6c02 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:09.453892 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb28e586-2e87-422e-9380-4f5e271c6c02 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:09.486322 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8bac99-8985-4f67-9310-590150cf5d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:09.486951 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8bac99-8985-4f67-9310-590150cf5d7f None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:09.487232 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8bac99-8985-4f67-9310-590150cf5d7f None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:09.487505 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8bac99-8985-4f67-9310-590150cf5d7f None admin] PATH_INFO: `/v3/projects/fedb3641c9d143aebb204db3ec4897f7` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:09.488649 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8bac99-8985-4f67-9310-590150cf5d7f None admin] RBAC: Authorizing `identity:get_project(project_id=fedb3641c9d143aebb204db3ec4897f7)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:09.490449 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8bac99-8985-4f67-9310-590150cf5d7f None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:09.492124 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 571/1152] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Aug 6 01:22:09 2022] GET /identity/v3/projects/fedb3641c9d143aebb204db3ec4897f7 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:22:09.498683 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3393ec72-45ed-4847-b148-95284bf3aa1a None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:09.594984 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3393ec72-45ed-4847-b148-95284bf3aa1a None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:09.628358 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d10391-2cf6-4101-b834-1d0dc75ffe9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:09.629008 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d10391-2cf6-4101-b834-1d0dc75ffe9a None admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:09.629323 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d10391-2cf6-4101-b834-1d0dc75ffe9a None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:09.629602 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d10391-2cf6-4101-b834-1d0dc75ffe9a None admin] PATH_INFO: `/v3/projects/fedb3641c9d143aebb204db3ec4897f7/users/34d78e0cfdd3427d95df409f889cde2e/roles/4559b9a0837249f895a13c70096f8d38` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:09.631530 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d10391-2cf6-4101-b834-1d0dc75ffe9a None admin] RBAC: Authorizing `identity:create_grant(project_id=fedb3641c9d143aebb204db3ec4897f7, user_id=34d78e0cfdd3427d95df409f889cde2e, role_id=4559b9a0837249f895a13c70096f8d38)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:09.632951 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d10391-2cf6-4101-b834-1d0dc75ffe9a None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:09.646478 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 582/1153] 10.222.0.26 () {58 vars in 1439 bytes} [Sat Aug 6 01:22:09 2022] PUT /identity/v3/projects/fedb3641c9d143aebb204db3ec4897f7/users/34d78e0cfdd3427d95df409f889cde2e/roles/4559b9a0837249f895a13c70096f8d38 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:22:11.633762 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa141c2-231d-4ba0-ade1-8daa6dfd999c None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:11.634302 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa141c2-231d-4ba0-ade1-8daa6dfd999c None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:11.634302 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa141c2-231d-4ba0-ade1-8daa6dfd999c None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:11.635676 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 572/1154] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:22:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:22:11.640835 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01728ae-50a0-4582-8da1-ea8b3525e27d None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:11.641297 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01728ae-50a0-4582-8da1-ea8b3525e27d None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:11.641406 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01728ae-50a0-4582-8da1-ea8b3525e27d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:11.659637 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-d01728ae-50a0-4582-8da1-ea8b3525e27d None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:22:11.660757 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-d01728ae-50a0-4582-8da1-ea8b3525e27d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:11.662782 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:22:11.728273 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 583/1155] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:22:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:22:11.751338 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa141c2-231d-4ba0-ade1-8daa6dfd999c None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:11.781946 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:22:11.852544 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa141c2-231d-4ba0-ade1-8daa6dfd999c None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:11.884474 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae8a35c-fc00-4717-9657-1c8922e4ef4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:11.885123 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae8a35c-fc00-4717-9657-1c8922e4ef4d None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:11.885403 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae8a35c-fc00-4717-9657-1c8922e4ef4d None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:11.885676 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae8a35c-fc00-4717-9657-1c8922e4ef4d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:11.895722 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae8a35c-fc00-4717-9657-1c8922e4ef4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:22:11.901384 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-7ae8a35c-fc00-4717-9657-1c8922e4ef4d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:22:11.901384 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:22:11.901384 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:22:11.901384 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:22:11.901384 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:22:11.901384 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:22:11.901384 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:22:11.901384 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:22:11.901384 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:22:11.901384 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:22:11.901384 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:22:11.901384 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:22:11.901384 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:22:11.901384 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:22:11.901384 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:22:11.901384 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:22:11.902470 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:22:11.902470 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:22:11.902470 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:22:11.902470 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:22:11.902470 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:22:11.902470 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:22:11.902470 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:22:11.902470 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:22:11.902470 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:22:11.902470 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:22:11.902470 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:22:11.902470 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:22:11.902470 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:22:11.902470 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:22:11.902470 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:22:11.902470 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:22:11.903476 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:22:11.903476 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:22:11.903476 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:22:11.903476 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:22:11.903476 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:22:11.903476 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:22:11.903476 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:22:11.903476 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:22:11.903476 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:22:11.903476 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:22:11.903476 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:22:11.903476 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:22:11.903476 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 573/1156] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Aug 6 01:22:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:22:11.908419 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01728ae-50a0-4582-8da1-ea8b3525e27d None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:12.004111 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01728ae-50a0-4582-8da1-ea8b3525e27d None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:12.037308 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc18e10-aa69-40c8-814a-57d296f49ff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:12.037913 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc18e10-aa69-40c8-814a-57d296f49ff2 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:12.038191 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc18e10-aa69-40c8-814a-57d296f49ff2 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:12.038462 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc18e10-aa69-40c8-814a-57d296f49ff2 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:12.039167 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc18e10-aa69-40c8-814a-57d296f49ff2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:22:12.039485 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc18e10-aa69-40c8-814a-57d296f49ff2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:12.040904 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc18e10-aa69-40c8-814a-57d296f49ff2 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:12.045787 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:22:12.049647 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 584/1157] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Aug 6 01:22:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:22:12.055050 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae8a35c-fc00-4717-9657-1c8922e4ef4d None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:12.143338 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:22:12.153308 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae8a35c-fc00-4717-9657-1c8922e4ef4d None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:12.185269 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4820a1b-e424-4c5c-862f-d5af1891f441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:12.185896 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4820a1b-e424-4c5c-862f-d5af1891f441 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:12.186181 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4820a1b-e424-4c5c-862f-d5af1891f441 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:12.186459 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4820a1b-e424-4c5c-862f-d5af1891f441 None admin] PATH_INFO: `/v3/users/34d78e0cfdd3427d95df409f889cde2e` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:12.187606 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4820a1b-e424-4c5c-862f-d5af1891f441 None admin] RBAC: Authorizing `identity:get_user(user_id=34d78e0cfdd3427d95df409f889cde2e)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:12.189012 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4820a1b-e424-4c5c-862f-d5af1891f441 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:12.190619 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 574/1158] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Aug 6 01:22:12 2022] GET /identity/v3/users/34d78e0cfdd3427d95df409f889cde2e => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:22:12.196859 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc18e10-aa69-40c8-814a-57d296f49ff2 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:12.292948 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc18e10-aa69-40c8-814a-57d296f49ff2 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:12.325399 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38816867-9ba0-4c53-80f1-feb9b3619580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:12.325997 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38816867-9ba0-4c53-80f1-feb9b3619580 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:12.326275 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38816867-9ba0-4c53-80f1-feb9b3619580 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:12.326546 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38816867-9ba0-4c53-80f1-feb9b3619580 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:12.388450 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:22:12.391129 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38816867-9ba0-4c53-80f1-feb9b3619580 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:12.392877 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38816867-9ba0-4c53-80f1-feb9b3619580 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:12.489644 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 585/1159] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Aug 6 01:22:12 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 06 01:22:12.495568 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4820a1b-e424-4c5c-862f-d5af1891f441 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:12.592334 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4820a1b-e424-4c5c-862f-d5af1891f441 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:12.624997 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3388d65c-a70e-4f62-8e17-18eb793511b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:12.625650 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3388d65c-a70e-4f62-8e17-18eb793511b4 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:12.625935 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3388d65c-a70e-4f62-8e17-18eb793511b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:12.626215 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3388d65c-a70e-4f62-8e17-18eb793511b4 None admin] PATH_INFO: `/v3/projects/fedb3641c9d143aebb204db3ec4897f7` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:12.627365 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3388d65c-a70e-4f62-8e17-18eb793511b4 None admin] RBAC: Authorizing `identity:get_project(project_id=fedb3641c9d143aebb204db3ec4897f7)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:12.628788 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3388d65c-a70e-4f62-8e17-18eb793511b4 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:12.630451 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 575/1160] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Aug 6 01:22:12 2022] GET /identity/v3/projects/fedb3641c9d143aebb204db3ec4897f7 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:22:12.637367 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38816867-9ba0-4c53-80f1-feb9b3619580 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:12.729691 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:22:12.732692 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38816867-9ba0-4c53-80f1-feb9b3619580 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:12.766301 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed18a76b-6cb3-4f9b-86dd-dea20df68db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:12.766909 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed18a76b-6cb3-4f9b-86dd-dea20df68db1 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:12.767196 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed18a76b-6cb3-4f9b-86dd-dea20df68db1 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:12.767472 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed18a76b-6cb3-4f9b-86dd-dea20df68db1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:12.768248 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed18a76b-6cb3-4f9b-86dd-dea20df68db1 None admin] RBAC: Adding query filter params (role.id=4559b9a0837249f895a13c70096f8d38, scope.project.id=fedb3641c9d143aebb204db3ec4897f7, user.id=34d78e0cfdd3427d95df409f889cde2e) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:22:12.768557 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed18a76b-6cb3-4f9b-86dd-dea20df68db1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:12.770011 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed18a76b-6cb3-4f9b-86dd-dea20df68db1 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:12.776211 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 586/1161] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Aug 6 01:22:12 2022] GET /identity/v3/role_assignments?user.id=34d78e0cfdd3427d95df409f889cde2e&scope.project.id=fedb3641c9d143aebb204db3ec4897f7&role.id=4559b9a0837249f895a13c70096f8d38 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:22:14.774213 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933c1032-2ec2-4afd-a7ee-1f4d0885d177 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:14.774766 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933c1032-2ec2-4afd-a7ee-1f4d0885d177 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:14.774766 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933c1032-2ec2-4afd-a7ee-1f4d0885d177 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:14.776139 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 576/1162] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:22:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:22:14.781265 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aca1d25-08a5-4184-b749-36617452a261 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:14.781682 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aca1d25-08a5-4184-b749-36617452a261 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:14.781824 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aca1d25-08a5-4184-b749-36617452a261 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:14.799949 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-1aca1d25-08a5-4184-b749-36617452a261 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:22:14.801168 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-1aca1d25-08a5-4184-b749-36617452a261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:14.865975 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 587/1163] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:22:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:22:14.889135 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933c1032-2ec2-4afd-a7ee-1f4d0885d177 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:14.916652 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:22:14.992029 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933c1032-2ec2-4afd-a7ee-1f4d0885d177 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:15.025500 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39204c37-9dff-473a-9514-e8e465b57006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:15.026053 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39204c37-9dff-473a-9514-e8e465b57006 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:15.026339 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39204c37-9dff-473a-9514-e8e465b57006 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:15.026618 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39204c37-9dff-473a-9514-e8e465b57006 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:15.091189 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39204c37-9dff-473a-9514-e8e465b57006 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:15.092629 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39204c37-9dff-473a-9514-e8e465b57006 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:15.117359 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:22:15.188114 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 577/1164] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Aug 6 01:22:14 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 06 01:22:15.194091 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aca1d25-08a5-4184-b749-36617452a261 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:15.286466 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:22:15.293719 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aca1d25-08a5-4184-b749-36617452a261 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:15.327038 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4c14de-9737-4458-b67a-5d125500d908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:15.327680 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4c14de-9737-4458-b67a-5d125500d908 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:15.327967 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4c14de-9737-4458-b67a-5d125500d908 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:15.328247 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4c14de-9737-4458-b67a-5d125500d908 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:15.334816 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4c14de-9737-4458-b67a-5d125500d908 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:15.336269 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4c14de-9737-4458-b67a-5d125500d908 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:15.347816 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-3c4c14de-9737-4458-b67a-5d125500d908 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 06 01:22:15.347816 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:22:15.347816 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 06 01:22:15.347816 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 06 01:22:15.347816 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 01:22:15.347816 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 01:22:15.347816 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 01:22:15.347816 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 01:22:15.347816 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 06 01:22:15.347816 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:22:15.347816 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 01:22:15.347816 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:22:15.347816 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:22:15.347816 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:22:15.347816 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:22:15.347816 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:22:15.347816 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:22:15.349011 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:22:15.349011 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:22:15.349011 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:22:15.349011 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:22:15.349011 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:22:15.349011 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:22:15.349011 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:22:15.349011 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:22:15.349011 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:22:15.349011 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:22:15.349011 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:22:15.349011 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:22:15.349011 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:22:15.349011 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:22:15.349011 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:22:15.349011 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:22:15.350083 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:22:15.350083 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:22:15.350083 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:22:15.350083 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:22:15.350083 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:22:15.350083 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:22:15.350083 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:22:15.350083 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:22:15.350083 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:22:15.350083 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:22:15.350083 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 06 01:22:15.350083 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:22:15.350083 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 06 01:22:15.350083 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:22:15.350083 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 588/1165] 10.222.0.26 () {56 vars in 1117 bytes} [Sat Aug 6 01:22:15 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:22:15.355292 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39204c37-9dff-473a-9514-e8e465b57006 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:15.450529 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39204c37-9dff-473a-9514-e8e465b57006 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:15.475037 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:22:15.490466 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64680469-6a19-4a8b-84d3-1d211cc018fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:15.491084 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64680469-6a19-4a8b-84d3-1d211cc018fa None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:15.491367 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64680469-6a19-4a8b-84d3-1d211cc018fa None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:15.491641 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64680469-6a19-4a8b-84d3-1d211cc018fa None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:15.492325 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64680469-6a19-4a8b-84d3-1d211cc018fa None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:22:15.493646 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64680469-6a19-4a8b-84d3-1d211cc018fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:15.495022 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64680469-6a19-4a8b-84d3-1d211cc018fa None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:15.502066 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 578/1166] 10.222.0.26 () {56 vars in 1115 bytes} [Sat Aug 6 01:22:15 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:22:17.505996 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75db02b-08e0-47fd-8ebf-fe366945618d None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:17.506544 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75db02b-08e0-47fd-8ebf-fe366945618d None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:17.506544 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75db02b-08e0-47fd-8ebf-fe366945618d None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:17.507893 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 589/1167] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:22:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:22:17.513830 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6009c2-d62a-48e6-ade2-b85754cbb737 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:17.514226 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6009c2-d62a-48e6-ade2-b85754cbb737 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:17.514399 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6009c2-d62a-48e6-ade2-b85754cbb737 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:17.531945 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-8c6009c2-d62a-48e6-ade2-b85754cbb737 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:22:17.533165 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-8c6009c2-d62a-48e6-ade2-b85754cbb737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:17.598494 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 579/1168] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:22:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:22:17.622127 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75db02b-08e0-47fd-8ebf-fe366945618d None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:17.721886 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75db02b-08e0-47fd-8ebf-fe366945618d None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:17.754467 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a0881e-7001-4fab-87e8-5c1f0668818d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:17.755070 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a0881e-7001-4fab-87e8-5c1f0668818d None admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:17.755351 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a0881e-7001-4fab-87e8-5c1f0668818d None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:17.755627 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a0881e-7001-4fab-87e8-5c1f0668818d None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:17.756336 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a0881e-7001-4fab-87e8-5c1f0668818d None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:17.757716 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a0881e-7001-4fab-87e8-5c1f0668818d None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:17.773403 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 590/1169] 10.222.0.26 () {60 vars in 1138 bytes} [Sat Aug 6 01:22:17 2022] POST /identity/v3/domains => generated 267 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:22:19.791264 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caaa7c3-0e34-4a03-a44a-3b45e8d0546e None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:19.791813 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caaa7c3-0e34-4a03-a44a-3b45e8d0546e None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:19.791813 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caaa7c3-0e34-4a03-a44a-3b45e8d0546e None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:19.793178 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 580/1170] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:22:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:22:19.798271 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98afff55-3f7e-4c84-8204-950af4ca998e None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:19.798686 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98afff55-3f7e-4c84-8204-950af4ca998e None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:19.798832 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98afff55-3f7e-4c84-8204-950af4ca998e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:19.816497 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-98afff55-3f7e-4c84-8204-950af4ca998e None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:22:19.817719 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-98afff55-3f7e-4c84-8204-950af4ca998e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:19.826802 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:22:19.884468 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 591/1171] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:22:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:22:19.908135 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caaa7c3-0e34-4a03-a44a-3b45e8d0546e None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:20.007347 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caaa7c3-0e34-4a03-a44a-3b45e8d0546e None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:20.027064 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:22:20.042232 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaaed31-59fc-4fad-ba11-bb65d1a6535d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:20.042809 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaaed31-59fc-4fad-ba11-bb65d1a6535d None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:20.043095 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaaed31-59fc-4fad-ba11-bb65d1a6535d None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:20.043389 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaaed31-59fc-4fad-ba11-bb65d1a6535d None admin] PATH_INFO: `/v3/domains/b0afa0ba19f946ed8f072e968ba4ee14` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:20.053578 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eaaed31-59fc-4fad-ba11-bb65d1a6535d None admin] RBAC: Authorizing `identity:get_domain(domain_id=b0afa0ba19f946ed8f072e968ba4ee14)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:20.055014 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eaaed31-59fc-4fad-ba11-bb65d1a6535d None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:20.056625 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 581/1172] 10.222.0.26 () {56 vars in 1183 bytes} [Sat Aug 6 01:22:19 2022] GET /identity/v3/domains/b0afa0ba19f946ed8f072e968ba4ee14 => generated 267 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:22:20.062407 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98afff55-3f7e-4c84-8204-950af4ca998e None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:20.157800 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98afff55-3f7e-4c84-8204-950af4ca998e None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:20.189486 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82779249-dc05-4eab-9fc8-3268d978cf8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:20.190089 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82779249-dc05-4eab-9fc8-3268d978cf8f None admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:20.190368 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82779249-dc05-4eab-9fc8-3268d978cf8f None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:20.190639 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82779249-dc05-4eab-9fc8-3268d978cf8f None admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:20.191316 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82779249-dc05-4eab-9fc8-3268d978cf8f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:20.192719 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82779249-dc05-4eab-9fc8-3268d978cf8f None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:20.207770 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:22:20.221385 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 592/1173] 10.222.0.26 () {60 vars in 1141 bytes} [Sat Aug 6 01:22:20 2022] POST /identity/v3/projects => generated 361 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:22:22.292876 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbb848a-4112-4fc8-9b0c-8f88bd7ef44c None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:22.293453 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbb848a-4112-4fc8-9b0c-8f88bd7ef44c None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:22.293453 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbb848a-4112-4fc8-9b0c-8f88bd7ef44c None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:22.294826 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 582/1174] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:22:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:22:22.299826 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba19c0c-ff5e-4a68-b29c-8bae432f8ef6 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:22.300252 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba19c0c-ff5e-4a68-b29c-8bae432f8ef6 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:22.300378 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba19c0c-ff5e-4a68-b29c-8bae432f8ef6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:22.319307 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-2ba19c0c-ff5e-4a68-b29c-8bae432f8ef6 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:22:22.321206 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-2ba19c0c-ff5e-4a68-b29c-8bae432f8ef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:22.386453 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 593/1175] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:22:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:22:22.410459 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbb848a-4112-4fc8-9b0c-8f88bd7ef44c None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:22.510436 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbb848a-4112-4fc8-9b0c-8f88bd7ef44c None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:22.543462 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3ded0c-1a98-4e55-b2af-77fce9fe3524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:22.544036 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3ded0c-1a98-4e55-b2af-77fce9fe3524 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:22.544324 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3ded0c-1a98-4e55-b2af-77fce9fe3524 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:22.544603 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3ded0c-1a98-4e55-b2af-77fce9fe3524 None admin] PATH_INFO: `/v3/domains/b0afa0ba19f946ed8f072e968ba4ee14` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:22.545772 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3ded0c-1a98-4e55-b2af-77fce9fe3524 None admin] RBAC: Authorizing `identity:get_domain(domain_id=b0afa0ba19f946ed8f072e968ba4ee14)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:22.547193 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3ded0c-1a98-4e55-b2af-77fce9fe3524 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:22.548769 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 583/1176] 10.222.0.26 () {56 vars in 1183 bytes} [Sat Aug 6 01:22:22 2022] GET /identity/v3/domains/b0afa0ba19f946ed8f072e968ba4ee14 => generated 267 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:22:22.555568 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba19c0c-ff5e-4a68-b29c-8bae432f8ef6 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:22.655815 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba19c0c-ff5e-4a68-b29c-8bae432f8ef6 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:22.679932 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:22:22.688818 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aba2180-eb62-4702-9903-21093bab6d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:22.689466 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aba2180-eb62-4702-9903-21093bab6d95 None admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:22.689749 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aba2180-eb62-4702-9903-21093bab6d95 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:22.690021 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aba2180-eb62-4702-9903-21093bab6d95 None admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:22.690695 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aba2180-eb62-4702-9903-21093bab6d95 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:22.692123 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aba2180-eb62-4702-9903-21093bab6d95 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:22.713542 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 594/1177] 10.222.0.26 () {60 vars in 1132 bytes} [Sat Aug 6 01:22:22 2022] POST /identity/v3/users => generated 311 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:22:24.763547 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f694e724-9b17-4a8d-8f3f-1618f1a63fd0 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:24.764116 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f694e724-9b17-4a8d-8f3f-1618f1a63fd0 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:24.764116 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f694e724-9b17-4a8d-8f3f-1618f1a63fd0 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:24.765498 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 584/1178] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:22:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:22:24.770654 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf1ff4f-8cd9-436d-b0ae-faee96eef2e5 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:24.771052 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf1ff4f-8cd9-436d-b0ae-faee96eef2e5 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:24.771214 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf1ff4f-8cd9-436d-b0ae-faee96eef2e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:24.789589 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-5bf1ff4f-8cd9-436d-b0ae-faee96eef2e5 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:22:24.790989 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-5bf1ff4f-8cd9-436d-b0ae-faee96eef2e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:24.856781 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 595/1179] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:22:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:22:24.879575 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f694e724-9b17-4a8d-8f3f-1618f1a63fd0 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:24.931466 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:22:24.982586 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f694e724-9b17-4a8d-8f3f-1618f1a63fd0 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:25.015418 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef87dbe-e721-4722-96d9-bc4598dd921d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:25.015985 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef87dbe-e721-4722-96d9-bc4598dd921d None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:25.016266 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef87dbe-e721-4722-96d9-bc4598dd921d None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:25.016539 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef87dbe-e721-4722-96d9-bc4598dd921d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:25.027211 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef87dbe-e721-4722-96d9-bc4598dd921d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:22:25.032008 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-4ef87dbe-e721-4722-96d9-bc4598dd921d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:22:25.032008 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:22:25.032008 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:22:25.032008 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:22:25.032008 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:22:25.032008 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:22:25.032008 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:22:25.032008 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:22:25.032008 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:22:25.032008 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:22:25.032008 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:22:25.032008 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:22:25.032008 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:22:25.032008 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:22:25.032008 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:22:25.032008 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:22:25.033217 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:22:25.033217 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:22:25.033217 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:22:25.033217 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:22:25.033217 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:22:25.033217 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:22:25.033217 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:22:25.033217 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:22:25.033217 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:22:25.033217 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:22:25.033217 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:22:25.033217 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:22:25.033217 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:22:25.033217 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:22:25.033217 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:22:25.033217 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:22:25.034320 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:22:25.034320 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:22:25.034320 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:22:25.034320 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:22:25.034320 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:22:25.034320 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:22:25.034320 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:22:25.034320 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:22:25.034320 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:22:25.034320 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:22:25.034320 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:22:25.034320 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:22:25.034320 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 585/1180] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Aug 6 01:22:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:22:25.039807 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf1ff4f-8cd9-436d-b0ae-faee96eef2e5 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:25.136582 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf1ff4f-8cd9-436d-b0ae-faee96eef2e5 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:25.149653 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:22:25.170841 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefac174-94d4-4665-8e8e-eaa860fdb271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:25.171453 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefac174-94d4-4665-8e8e-eaa860fdb271 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:25.171736 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefac174-94d4-4665-8e8e-eaa860fdb271 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:25.172014 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefac174-94d4-4665-8e8e-eaa860fdb271 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:25.172739 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefac174-94d4-4665-8e8e-eaa860fdb271 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:22:25.173063 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefac174-94d4-4665-8e8e-eaa860fdb271 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:25.174435 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefac174-94d4-4665-8e8e-eaa860fdb271 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:25.182266 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 596/1181] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Aug 6 01:22:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:22:25.188008 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef87dbe-e721-4722-96d9-bc4598dd921d None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:25.285957 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef87dbe-e721-4722-96d9-bc4598dd921d None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:25.294572 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:22:25.319359 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae200be-f36a-450c-8642-c479c8a7d6d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:25.319981 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae200be-f36a-450c-8642-c479c8a7d6d5 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:25.320288 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae200be-f36a-450c-8642-c479c8a7d6d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:25.321127 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae200be-f36a-450c-8642-c479c8a7d6d5 None admin] PATH_INFO: `/v3/users/541dbfead05744fdaf1d6dd29abcef41` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:25.333832 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae200be-f36a-450c-8642-c479c8a7d6d5 None admin] RBAC: Authorizing `identity:get_user(user_id=541dbfead05744fdaf1d6dd29abcef41)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:25.335267 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae200be-f36a-450c-8642-c479c8a7d6d5 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:25.336968 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 586/1182] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Aug 6 01:22:25 2022] GET /identity/v3/users/541dbfead05744fdaf1d6dd29abcef41 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:22:25.342687 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefac174-94d4-4665-8e8e-eaa860fdb271 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:25.441719 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefac174-94d4-4665-8e8e-eaa860fdb271 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:25.473651 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed40652-1ed7-4056-80a2-00f3d3165b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:25.474259 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed40652-1ed7-4056-80a2-00f3d3165b5e None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:25.474538 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed40652-1ed7-4056-80a2-00f3d3165b5e None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:25.474806 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed40652-1ed7-4056-80a2-00f3d3165b5e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:25.515694 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:22:25.540600 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed40652-1ed7-4056-80a2-00f3d3165b5e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:25.542002 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed40652-1ed7-4056-80a2-00f3d3165b5e None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:25.636473 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 597/1183] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Aug 6 01:22:25 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 06 01:22:25.642534 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae200be-f36a-450c-8642-c479c8a7d6d5 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:25.738398 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae200be-f36a-450c-8642-c479c8a7d6d5 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:25.771490 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491e41e2-ce8a-4ad7-8382-d4624de6780f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:25.772120 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491e41e2-ce8a-4ad7-8382-d4624de6780f None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:25.772404 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491e41e2-ce8a-4ad7-8382-d4624de6780f None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:25.772683 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491e41e2-ce8a-4ad7-8382-d4624de6780f None admin] PATH_INFO: `/v3/projects/2ffb3603645140309bd0e040b0c22ba4` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:25.774375 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491e41e2-ce8a-4ad7-8382-d4624de6780f None admin] RBAC: Authorizing `identity:get_project(project_id=2ffb3603645140309bd0e040b0c22ba4)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:25.775791 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491e41e2-ce8a-4ad7-8382-d4624de6780f None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:25.777482 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 587/1184] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Aug 6 01:22:25 2022] GET /identity/v3/projects/2ffb3603645140309bd0e040b0c22ba4 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:22:25.783463 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed40652-1ed7-4056-80a2-00f3d3165b5e None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:25.855950 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:22:25.880043 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed40652-1ed7-4056-80a2-00f3d3165b5e None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:25.913137 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11b5dd4-eb37-4adc-8399-d453ed8a5fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:25.913744 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11b5dd4-eb37-4adc-8399-d453ed8a5fb9 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:25.914725 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11b5dd4-eb37-4adc-8399-d453ed8a5fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:25.915009 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11b5dd4-eb37-4adc-8399-d453ed8a5fb9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:25.915791 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11b5dd4-eb37-4adc-8399-d453ed8a5fb9 None admin] RBAC: Adding query filter params (role.id=4559b9a0837249f895a13c70096f8d38, scope.project.id=2ffb3603645140309bd0e040b0c22ba4, user.id=541dbfead05744fdaf1d6dd29abcef41) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:22:25.916102 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11b5dd4-eb37-4adc-8399-d453ed8a5fb9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:25.917551 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11b5dd4-eb37-4adc-8399-d453ed8a5fb9 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:25.922787 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 598/1185] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Aug 6 01:22:25 2022] GET /identity/v3/role_assignments?user.id=541dbfead05744fdaf1d6dd29abcef41&scope.project.id=2ffb3603645140309bd0e040b0c22ba4&role.id=4559b9a0837249f895a13c70096f8d38 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:22:27.921589 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ed9353-4b2b-426e-ab82-412aa4ddd648 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:27.922121 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ed9353-4b2b-426e-ab82-412aa4ddd648 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:27.922121 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ed9353-4b2b-426e-ab82-412aa4ddd648 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:27.923494 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 588/1186] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:22:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:22:27.928579 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eefe4e3-1cbf-4cc4-97e1-577055b6f8a9 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:27.928961 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eefe4e3-1cbf-4cc4-97e1-577055b6f8a9 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:27.929177 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eefe4e3-1cbf-4cc4-97e1-577055b6f8a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:27.947653 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-4eefe4e3-1cbf-4cc4-97e1-577055b6f8a9 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:22:27.948835 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-4eefe4e3-1cbf-4cc4-97e1-577055b6f8a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:28.014124 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 599/1187] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:22:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:22:28.036523 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ed9353-4b2b-426e-ab82-412aa4ddd648 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:28.072339 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:22:28.137598 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ed9353-4b2b-426e-ab82-412aa4ddd648 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:28.170719 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28339afc-e5bb-4b49-ab46-f7ddb3898e8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:28.171343 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28339afc-e5bb-4b49-ab46-f7ddb3898e8f None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:28.171672 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28339afc-e5bb-4b49-ab46-f7ddb3898e8f None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:28.171900 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28339afc-e5bb-4b49-ab46-f7ddb3898e8f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:28.181973 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28339afc-e5bb-4b49-ab46-f7ddb3898e8f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:22:28.186859 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-28339afc-e5bb-4b49-ab46-f7ddb3898e8f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:22:28.186859 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:22:28.186859 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:22:28.186859 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:22:28.186859 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:22:28.186859 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:22:28.186859 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:22:28.186859 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:22:28.186859 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:22:28.186859 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:22:28.186859 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:22:28.186859 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:22:28.186859 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:22:28.186859 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:22:28.186859 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:22:28.186859 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:22:28.187933 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:22:28.187933 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:22:28.187933 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:22:28.187933 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:22:28.187933 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:22:28.187933 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:22:28.187933 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:22:28.187933 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:22:28.187933 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:22:28.187933 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:22:28.187933 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:22:28.187933 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:22:28.187933 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:22:28.187933 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:22:28.187933 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:22:28.187933 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:22:28.189149 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:22:28.189149 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:22:28.189149 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:22:28.189149 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:22:28.189149 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:22:28.189149 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:22:28.189149 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:22:28.189149 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:22:28.189149 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:22:28.189149 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:22:28.189149 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:22:28.189149 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:22:28.189149 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 589/1188] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Aug 6 01:22:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:22:28.194024 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eefe4e3-1cbf-4cc4-97e1-577055b6f8a9 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:28.258787 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:22:28.291058 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eefe4e3-1cbf-4cc4-97e1-577055b6f8a9 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:28.324234 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1458d4d-3f5a-4f3e-82c0-8d9f05da6c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:28.324848 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1458d4d-3f5a-4f3e-82c0-8d9f05da6c76 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:28.325154 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1458d4d-3f5a-4f3e-82c0-8d9f05da6c76 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:28.325426 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1458d4d-3f5a-4f3e-82c0-8d9f05da6c76 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:28.326137 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1458d4d-3f5a-4f3e-82c0-8d9f05da6c76 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:22:28.326438 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1458d4d-3f5a-4f3e-82c0-8d9f05da6c76 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:28.327785 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1458d4d-3f5a-4f3e-82c0-8d9f05da6c76 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:28.335559 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 600/1189] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Aug 6 01:22:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:22:28.341428 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28339afc-e5bb-4b49-ab46-f7ddb3898e8f None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:28.434882 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:22:28.441956 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28339afc-e5bb-4b49-ab46-f7ddb3898e8f None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:28.474285 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4722e57a-4001-4dbc-a78e-c2cc6f61d88a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:28.474913 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4722e57a-4001-4dbc-a78e-c2cc6f61d88a None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:28.475195 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4722e57a-4001-4dbc-a78e-c2cc6f61d88a None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:28.475473 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4722e57a-4001-4dbc-a78e-c2cc6f61d88a None admin] PATH_INFO: `/v3/users/541dbfead05744fdaf1d6dd29abcef41` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:28.477439 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4722e57a-4001-4dbc-a78e-c2cc6f61d88a None admin] RBAC: Authorizing `identity:get_user(user_id=541dbfead05744fdaf1d6dd29abcef41)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:28.478946 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4722e57a-4001-4dbc-a78e-c2cc6f61d88a None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:28.480538 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 590/1190] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Aug 6 01:22:28 2022] GET /identity/v3/users/541dbfead05744fdaf1d6dd29abcef41 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:22:28.486277 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1458d4d-3f5a-4f3e-82c0-8d9f05da6c76 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:28.582203 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1458d4d-3f5a-4f3e-82c0-8d9f05da6c76 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:28.609912 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:22:28.615853 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24d6f5c-6234-4daf-a7b1-ec0511fedb7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:28.616489 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24d6f5c-6234-4daf-a7b1-ec0511fedb7c None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:28.616770 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24d6f5c-6234-4daf-a7b1-ec0511fedb7c None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:28.617641 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24d6f5c-6234-4daf-a7b1-ec0511fedb7c None admin] PATH_INFO: `/v3/projects/2ffb3603645140309bd0e040b0c22ba4` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:28.618865 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24d6f5c-6234-4daf-a7b1-ec0511fedb7c None admin] RBAC: Authorizing `identity:get_project(project_id=2ffb3603645140309bd0e040b0c22ba4)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:28.620283 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24d6f5c-6234-4daf-a7b1-ec0511fedb7c None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:28.621955 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 601/1191] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Aug 6 01:22:28 2022] GET /identity/v3/projects/2ffb3603645140309bd0e040b0c22ba4 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:22:28.627857 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4722e57a-4001-4dbc-a78e-c2cc6f61d88a None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:28.723005 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4722e57a-4001-4dbc-a78e-c2cc6f61d88a None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:28.756291 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643519f6-ed59-4da1-8a49-d606528bb31e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:28.756943 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643519f6-ed59-4da1-8a49-d606528bb31e None admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:28.757258 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643519f6-ed59-4da1-8a49-d606528bb31e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:28.757543 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643519f6-ed59-4da1-8a49-d606528bb31e None admin] PATH_INFO: `/v3/projects/2ffb3603645140309bd0e040b0c22ba4/users/541dbfead05744fdaf1d6dd29abcef41/roles/4559b9a0837249f895a13c70096f8d38` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:28.759580 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643519f6-ed59-4da1-8a49-d606528bb31e None admin] RBAC: Authorizing `identity:create_grant(project_id=2ffb3603645140309bd0e040b0c22ba4, user_id=541dbfead05744fdaf1d6dd29abcef41, role_id=4559b9a0837249f895a13c70096f8d38)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:28.760997 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643519f6-ed59-4da1-8a49-d606528bb31e None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:28.773091 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 591/1192] 10.222.0.26 () {58 vars in 1439 bytes} [Sat Aug 6 01:22:28 2022] PUT /identity/v3/projects/2ffb3603645140309bd0e040b0c22ba4/users/541dbfead05744fdaf1d6dd29abcef41/roles/4559b9a0837249f895a13c70096f8d38 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:22:30.802990 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1728327-45fb-4d52-8597-500b9f965500 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:30.803514 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1728327-45fb-4d52-8597-500b9f965500 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:30.803514 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1728327-45fb-4d52-8597-500b9f965500 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:30.805937 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 602/1193] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:22:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:22:30.811169 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cdf02f-1df9-494c-8b29-94df7734810a None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:30.811592 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cdf02f-1df9-494c-8b29-94df7734810a None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:30.811726 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cdf02f-1df9-494c-8b29-94df7734810a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:30.830436 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-c9cdf02f-1df9-494c-8b29-94df7734810a None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:22:30.831629 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-c9cdf02f-1df9-494c-8b29-94df7734810a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:30.848053 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:22:30.897823 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 592/1194] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:22:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:22:30.921307 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1728327-45fb-4d52-8597-500b9f965500 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:31.025962 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1728327-45fb-4d52-8597-500b9f965500 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:31.059244 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe8b19b-a342-408e-a192-9768197ba6b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:31.059816 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe8b19b-a342-408e-a192-9768197ba6b5 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:31.060100 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe8b19b-a342-408e-a192-9768197ba6b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:31.060380 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe8b19b-a342-408e-a192-9768197ba6b5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:31.070972 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe8b19b-a342-408e-a192-9768197ba6b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:22:31.075885 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-cbe8b19b-a342-408e-a192-9768197ba6b5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:22:31.075885 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:22:31.075885 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:22:31.075885 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:22:31.075885 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:22:31.075885 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:22:31.075885 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:22:31.075885 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:22:31.075885 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:22:31.075885 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:22:31.075885 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:22:31.075885 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:22:31.075885 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:22:31.075885 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:22:31.075885 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:22:31.075885 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:22:31.077141 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:22:31.077141 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:22:31.077141 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:22:31.077141 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:22:31.077141 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:22:31.077141 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:22:31.077141 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:22:31.077141 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:22:31.077141 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:22:31.077141 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:22:31.077141 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:22:31.077141 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:22:31.077141 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:22:31.077141 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:22:31.077141 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:22:31.077141 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:22:31.078282 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:22:31.078282 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:22:31.078282 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:22:31.078282 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:22:31.078282 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:22:31.078282 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:22:31.078282 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:22:31.078282 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:22:31.078282 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:22:31.078282 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:22:31.078282 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:22:31.078282 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:22:31.078282 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 603/1195] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Aug 6 01:22:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:22:31.083422 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cdf02f-1df9-494c-8b29-94df7734810a None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:31.178531 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cdf02f-1df9-494c-8b29-94df7734810a None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:31.212034 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b65f28-2a04-4fbd-9d8b-8776953f58f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:31.212638 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b65f28-2a04-4fbd-9d8b-8776953f58f6 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:31.212919 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b65f28-2a04-4fbd-9d8b-8776953f58f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:31.213245 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b65f28-2a04-4fbd-9d8b-8776953f58f6 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:31.213953 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b65f28-2a04-4fbd-9d8b-8776953f58f6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:22:31.214252 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b65f28-2a04-4fbd-9d8b-8776953f58f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:31.215607 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b65f28-2a04-4fbd-9d8b-8776953f58f6 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:31.223316 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 593/1196] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Aug 6 01:22:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:22:31.229531 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe8b19b-a342-408e-a192-9768197ba6b5 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:31.287056 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:22:31.327735 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe8b19b-a342-408e-a192-9768197ba6b5 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:31.360778 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7fbf0f-3739-45e8-89be-6a3e7b30786a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:31.361450 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7fbf0f-3739-45e8-89be-6a3e7b30786a None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:31.361735 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7fbf0f-3739-45e8-89be-6a3e7b30786a None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:31.362014 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7fbf0f-3739-45e8-89be-6a3e7b30786a None admin] PATH_INFO: `/v3/users/541dbfead05744fdaf1d6dd29abcef41` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:31.363178 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7fbf0f-3739-45e8-89be-6a3e7b30786a None admin] RBAC: Authorizing `identity:get_user(user_id=541dbfead05744fdaf1d6dd29abcef41)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:31.364613 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7fbf0f-3739-45e8-89be-6a3e7b30786a None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:31.366259 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 604/1197] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Aug 6 01:22:31 2022] GET /identity/v3/users/541dbfead05744fdaf1d6dd29abcef41 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:22:31.372662 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b65f28-2a04-4fbd-9d8b-8776953f58f6 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:31.388669 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:22:31.471141 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b65f28-2a04-4fbd-9d8b-8776953f58f6 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:31.504347 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5282463-0a6e-409b-bed9-70605b37ffd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:31.504961 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5282463-0a6e-409b-bed9-70605b37ffd0 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:31.505273 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5282463-0a6e-409b-bed9-70605b37ffd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:31.505555 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5282463-0a6e-409b-bed9-70605b37ffd0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:31.569982 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5282463-0a6e-409b-bed9-70605b37ffd0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:31.571610 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5282463-0a6e-409b-bed9-70605b37ffd0 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:31.585724 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:22:31.672236 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 594/1198] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Aug 6 01:22:31 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 06 01:22:31.678148 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7fbf0f-3739-45e8-89be-6a3e7b30786a None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:31.775181 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7fbf0f-3739-45e8-89be-6a3e7b30786a None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:31.799967 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:22:31.809336 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b487489-20ca-47a1-b2e0-3b0ef3c8110d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:31.809967 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b487489-20ca-47a1-b2e0-3b0ef3c8110d None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:31.810250 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b487489-20ca-47a1-b2e0-3b0ef3c8110d None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:31.810588 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b487489-20ca-47a1-b2e0-3b0ef3c8110d None admin] PATH_INFO: `/v3/projects/2ffb3603645140309bd0e040b0c22ba4` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:31.811754 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b487489-20ca-47a1-b2e0-3b0ef3c8110d None admin] RBAC: Authorizing `identity:get_project(project_id=2ffb3603645140309bd0e040b0c22ba4)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:31.813215 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b487489-20ca-47a1-b2e0-3b0ef3c8110d None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:31.814853 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 605/1199] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Aug 6 01:22:31 2022] GET /identity/v3/projects/2ffb3603645140309bd0e040b0c22ba4 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:22:31.820947 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5282463-0a6e-409b-bed9-70605b37ffd0 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:31.916155 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5282463-0a6e-409b-bed9-70605b37ffd0 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:31.929362 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:22:31.950100 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e15edf0-4d31-4625-8559-d8fac9520851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:31.950705 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e15edf0-4d31-4625-8559-d8fac9520851 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:31.950985 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e15edf0-4d31-4625-8559-d8fac9520851 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:31.951260 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e15edf0-4d31-4625-8559-d8fac9520851 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:31.952034 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e15edf0-4d31-4625-8559-d8fac9520851 None admin] RBAC: Adding query filter params (role.id=4559b9a0837249f895a13c70096f8d38, scope.project.id=2ffb3603645140309bd0e040b0c22ba4, user.id=541dbfead05744fdaf1d6dd29abcef41) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:22:31.952344 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e15edf0-4d31-4625-8559-d8fac9520851 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:31.953791 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e15edf0-4d31-4625-8559-d8fac9520851 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:31.959461 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 595/1200] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Aug 6 01:22:31 2022] GET /identity/v3/role_assignments?user.id=541dbfead05744fdaf1d6dd29abcef41&scope.project.id=2ffb3603645140309bd0e040b0c22ba4&role.id=4559b9a0837249f895a13c70096f8d38 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:22:41.804687 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:22:41.956887 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=11 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:27.065443 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bf79d9-1158-445d-8e5a-f2c187e3de5c None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:27.066370 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bf79d9-1158-445d-8e5a-f2c187e3de5c None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:27.066698 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bf79d9-1158-445d-8e5a-f2c187e3de5c None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:27.068099 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 606/1201] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:23:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:23:27.074446 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a535f1-f358-48dc-a549-32b2a564b465 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:27.074970 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a535f1-f358-48dc-a549-32b2a564b465 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:27.075059 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a535f1-f358-48dc-a549-32b2a564b465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:27.102806 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-50a535f1-f358-48dc-a549-32b2a564b465 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:23:27.102806 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-50a535f1-f358-48dc-a549-32b2a564b465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:27.184519 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 596/1202] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:23:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:23:27.212419 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bf79d9-1158-445d-8e5a-f2c187e3de5c None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:27.359768 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bf79d9-1158-445d-8e5a-f2c187e3de5c None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:27.412580 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dc118c-9daa-40d1-96cd-8eff81d7f4c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:27.413671 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dc118c-9daa-40d1-96cd-8eff81d7f4c6 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:27.413964 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dc118c-9daa-40d1-96cd-8eff81d7f4c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:27.414242 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dc118c-9daa-40d1-96cd-8eff81d7f4c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:27.427802 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dc118c-9daa-40d1-96cd-8eff81d7f4c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:27.430242 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dc118c-9daa-40d1-96cd-8eff81d7f4c6 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:27.451118 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-85dc118c-9daa-40d1-96cd-8eff81d7f4c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:23:27.451118 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:23:27.451118 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:23:27.451118 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:23:27.451118 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:23:27.451118 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:23:27.451118 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:23:27.451118 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:23:27.451118 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:23:27.451118 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:23:27.451118 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:23:27.451118 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:23:27.451118 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:23:27.451118 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:23:27.451118 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:23:27.452243 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:23:27.452243 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:23:27.452243 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:23:27.452243 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:23:27.452243 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:23:27.452243 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:23:27.452243 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:23:27.452243 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:23:27.452243 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:23:27.452243 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:23:27.452243 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:23:27.452243 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:23:27.452243 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:23:27.452243 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:23:27.452243 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:23:27.452243 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:23:27.452243 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:23:27.453347 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:23:27.453347 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:23:27.453347 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:23:27.453347 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:23:27.453347 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 607/1203] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:23:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:23:27.459091 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a535f1-f358-48dc-a549-32b2a564b465 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:27.635603 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a535f1-f358-48dc-a549-32b2a564b465 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:27.673941 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:27.684065 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55135fc2-94b6-4e79-b4de-33ff51902618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:27.684683 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55135fc2-94b6-4e79-b4de-33ff51902618 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:27.684964 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55135fc2-94b6-4e79-b4de-33ff51902618 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:27.685277 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55135fc2-94b6-4e79-b4de-33ff51902618 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:27.686007 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55135fc2-94b6-4e79-b4de-33ff51902618 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:23:27.686318 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55135fc2-94b6-4e79-b4de-33ff51902618 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:27.687773 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55135fc2-94b6-4e79-b4de-33ff51902618 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:27.701200 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 597/1204] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Aug 6 01:23:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:23:27.708684 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dc118c-9daa-40d1-96cd-8eff81d7f4c6 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:27.774795 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:27.834072 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dc118c-9daa-40d1-96cd-8eff81d7f4c6 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:27.875261 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff7f3e7-090b-48e1-9020-ae0173a6abc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:27.875893 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff7f3e7-090b-48e1-9020-ae0173a6abc3 None admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:27.876178 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff7f3e7-090b-48e1-9020-ae0173a6abc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:27.878976 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff7f3e7-090b-48e1-9020-ae0173a6abc3 None admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:27.878976 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff7f3e7-090b-48e1-9020-ae0173a6abc3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:27.878976 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff7f3e7-090b-48e1-9020-ae0173a6abc3 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:27.904333 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 608/1205] 10.222.0.26 () {60 vars in 1132 bytes} [Sat Aug 6 01:23:27 2022] POST /identity/v3/users => generated 251 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:23:30.087906 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e7f367-d578-4a88-91b5-684fa70c023c None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:30.088456 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e7f367-d578-4a88-91b5-684fa70c023c None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:30.088456 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e7f367-d578-4a88-91b5-684fa70c023c None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:30.090110 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 598/1206] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:23:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:23:30.096218 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa2821d-c7c3-4e7d-8020-84d3cfa88fd1 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:30.096708 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa2821d-c7c3-4e7d-8020-84d3cfa88fd1 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:30.096796 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa2821d-c7c3-4e7d-8020-84d3cfa88fd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:30.118243 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-afa2821d-c7c3-4e7d-8020-84d3cfa88fd1 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:23:30.119440 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-afa2821d-c7c3-4e7d-8020-84d3cfa88fd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:30.200931 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 609/1207] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:23:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:23:30.226887 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e7f367-d578-4a88-91b5-684fa70c023c None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:30.348494 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e7f367-d578-4a88-91b5-684fa70c023c None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:30.390768 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644a590d-5c48-476a-b73d-721c037c4180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:30.391621 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644a590d-5c48-476a-b73d-721c037c4180 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:30.392006 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644a590d-5c48-476a-b73d-721c037c4180 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:30.392388 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644a590d-5c48-476a-b73d-721c037c4180 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:30.409976 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644a590d-5c48-476a-b73d-721c037c4180 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:23:30.415739 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-644a590d-5c48-476a-b73d-721c037c4180 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 06 01:23:30.415739 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:23:30.415739 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:23:30.415739 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:23:30.415739 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:23:30.415739 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:23:30.415739 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:23:30.415739 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:23:30.415739 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:23:30.415739 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:23:30.415739 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:23:30.415739 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:23:30.415739 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:23:30.415739 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:23:30.415739 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:23:30.415739 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:23:30.416680 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:23:30.416680 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:23:30.416680 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:23:30.416680 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:23:30.416680 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:23:30.416680 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:23:30.416680 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:23:30.416680 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:23:30.416680 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:23:30.416680 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:23:30.416680 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:23:30.416680 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:23:30.416680 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:23:30.416680 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:23:30.416680 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:23:30.416680 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:23:30.417704 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:23:30.417704 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:23:30.417704 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:23:30.417704 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:23:30.417704 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:23:30.417704 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:23:30.417704 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:23:30.417704 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:23:30.417704 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:23:30.417704 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:23:30.417704 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 06 01:23:30.417704 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:23:30.417704 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 599/1208] 10.222.0.26 () {56 vars in 1102 bytes} [Sat Aug 6 01:23:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:23:30.422879 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa2821d-c7c3-4e7d-8020-84d3cfa88fd1 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:30.469228 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:30.548165 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa2821d-c7c3-4e7d-8020-84d3cfa88fd1 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:30.591002 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32a162f-a3bb-457a-bc3b-815cb22042aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:30.592038 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32a162f-a3bb-457a-bc3b-815cb22042aa None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:30.592533 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32a162f-a3bb-457a-bc3b-815cb22042aa None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:30.593071 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32a162f-a3bb-457a-bc3b-815cb22042aa None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:30.594028 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32a162f-a3bb-457a-bc3b-815cb22042aa None admin] RBAC: Adding query filter params (name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:23:30.594427 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32a162f-a3bb-457a-bc3b-815cb22042aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:30.595883 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32a162f-a3bb-457a-bc3b-815cb22042aa None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:30.605745 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 610/1209] 10.222.0.26 () {56 vars in 1103 bytes} [Sat Aug 6 01:23:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:23:30.611676 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644a590d-5c48-476a-b73d-721c037c4180 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:30.669149 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:30.732925 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644a590d-5c48-476a-b73d-721c037c4180 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:30.771156 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc72cc0-bdd8-4cb8-9857-d1f6286e42cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:30.771804 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc72cc0-bdd8-4cb8-9857-d1f6286e42cf None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:30.772782 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc72cc0-bdd8-4cb8-9857-d1f6286e42cf None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:30.773119 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc72cc0-bdd8-4cb8-9857-d1f6286e42cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:30.784185 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc72cc0-bdd8-4cb8-9857-d1f6286e42cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:30.785654 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc72cc0-bdd8-4cb8-9857-d1f6286e42cf None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:30.800435 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-5cc72cc0-bdd8-4cb8-9857-d1f6286e42cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:23:30.800435 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:23:30.800435 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:23:30.800435 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:23:30.800435 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:23:30.800435 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:23:30.800435 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:23:30.800435 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:23:30.800435 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:23:30.800435 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:23:30.800435 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:23:30.800435 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:23:30.800435 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:23:30.800435 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:23:30.800435 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:23:30.801592 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:23:30.801592 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:23:30.801592 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:23:30.801592 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:23:30.801592 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:23:30.801592 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:23:30.801592 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:23:30.801592 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:23:30.801592 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:23:30.801592 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:23:30.801592 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:23:30.801592 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:23:30.801592 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:23:30.801592 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:23:30.801592 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:23:30.801592 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:23:30.801592 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:23:30.802734 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:23:30.802734 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:23:30.802734 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:23:30.802734 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:23:30.806459 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 600/1210] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:23:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:23:30.807910 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32a162f-a3bb-457a-bc3b-815cb22042aa None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:30.921657 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:30.925534 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32a162f-a3bb-457a-bc3b-815cb22042aa None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:30.968881 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7e9525-6e86-447c-94ec-e33ef2789819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:30.969671 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7e9525-6e86-447c-94ec-e33ef2789819 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:30.969986 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7e9525-6e86-447c-94ec-e33ef2789819 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:30.970382 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7e9525-6e86-447c-94ec-e33ef2789819 None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:30.971105 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7e9525-6e86-447c-94ec-e33ef2789819 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:23:30.971482 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7e9525-6e86-447c-94ec-e33ef2789819 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:30.973135 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7e9525-6e86-447c-94ec-e33ef2789819 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:30.988870 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 611/1211] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Aug 6 01:23:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:23:30.995548 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc72cc0-bdd8-4cb8-9857-d1f6286e42cf None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:31.118082 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc72cc0-bdd8-4cb8-9857-d1f6286e42cf None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:31.120042 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:31.159101 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890e71b1-9799-4060-b478-0c5691f960ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:31.159871 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890e71b1-9799-4060-b478-0c5691f960ea None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:31.160272 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890e71b1-9799-4060-b478-0c5691f960ea None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:31.160559 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890e71b1-9799-4060-b478-0c5691f960ea None admin] PATH_INFO: `/v3/users/placement` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:31.169278 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890e71b1-9799-4060-b478-0c5691f960ea None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:31.170998 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890e71b1-9799-4060-b478-0c5691f960ea None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:31.185535 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-890e71b1-9799-4060-b478-0c5691f960ea None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 06 01:23:31.185535 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:23:31.185535 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:23:31.185535 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:23:31.185535 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:23:31.185535 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:23:31.185535 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:23:31.185535 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:23:31.185535 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:23:31.185535 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:23:31.185535 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:23:31.185535 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:23:31.185535 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 01:23:31.185535 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 01:23:31.185535 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 01:23:31.185535 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 01:23:31.186734 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:23:31.186734 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:23:31.186734 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 01:23:31.186734 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:23:31.186734 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 01:23:31.186734 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:23:31.186734 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:23:31.186734 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:23:31.186734 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:23:31.186734 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:23:31.186734 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:23:31.186734 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:23:31.186734 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:23:31.186734 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:23:31.186734 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:23:31.186734 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:23:31.186734 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:23:31.186734 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:23:31.192743 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:23:31.192743 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:23:31.192743 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 01:23:31.192743 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 01:23:31.192743 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 01:23:31.192743 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 01:23:31.192743 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 01:23:31.192743 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 01:23:31.192743 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 06 01:23:31.192743 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:23:31.192743 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 601/1212] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:23:30 2022] GET /identity/v3/users/placement => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:23:31.193533 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7e9525-6e86-447c-94ec-e33ef2789819 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:31.308156 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7e9525-6e86-447c-94ec-e33ef2789819 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:31.355906 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da31e576-2296-486a-9f28-63835f069380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:31.356560 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da31e576-2296-486a-9f28-63835f069380 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:31.358107 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da31e576-2296-486a-9f28-63835f069380 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:31.358107 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da31e576-2296-486a-9f28-63835f069380 None admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:31.358107 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da31e576-2296-486a-9f28-63835f069380 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:23:31.358448 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da31e576-2296-486a-9f28-63835f069380 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:31.359845 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da31e576-2296-486a-9f28-63835f069380 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:31.383783 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 612/1213] 10.222.0.26 () {56 vars in 1143 bytes} [Sat Aug 6 01:23:31 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:23:31.385011 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890e71b1-9799-4060-b478-0c5691f960ea None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:31.502992 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890e71b1-9799-4060-b478-0c5691f960ea None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:31.542820 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78e6453-8f36-467c-a7fd-34a016cd2ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:31.543514 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78e6453-8f36-467c-a7fd-34a016cd2ec7 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:31.543812 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78e6453-8f36-467c-a7fd-34a016cd2ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:31.544074 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78e6453-8f36-467c-a7fd-34a016cd2ec7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:31.581605 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:31.623911 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78e6453-8f36-467c-a7fd-34a016cd2ec7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:31.625565 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78e6453-8f36-467c-a7fd-34a016cd2ec7 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:31.744998 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 602/1214] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Aug 6 01:23:31 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 06 01:23:31.751264 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da31e576-2296-486a-9f28-63835f069380 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:31.777855 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:31.871776 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da31e576-2296-486a-9f28-63835f069380 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:31.907613 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e942f5c-d721-4e7a-b7ad-6dd790ee314b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:31.908281 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e942f5c-d721-4e7a-b7ad-6dd790ee314b None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:31.908546 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e942f5c-d721-4e7a-b7ad-6dd790ee314b None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:31.908825 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e942f5c-d721-4e7a-b7ad-6dd790ee314b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:31.919660 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e942f5c-d721-4e7a-b7ad-6dd790ee314b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:31.921138 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e942f5c-d721-4e7a-b7ad-6dd790ee314b None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:31.937513 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-7e942f5c-d721-4e7a-b7ad-6dd790ee314b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:23:31.937513 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:23:31.937513 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:23:31.937513 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:23:31.937513 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:23:31.937513 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:23:31.937513 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:23:31.937513 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:23:31.937513 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:23:31.937513 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:23:31.937513 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:23:31.937513 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:23:31.937513 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:23:31.937513 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:23:31.937513 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:23:31.938695 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:23:31.938695 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:23:31.938695 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:23:31.938695 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:23:31.938695 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:23:31.938695 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:23:31.938695 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:23:31.938695 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:23:31.938695 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:23:31.938695 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:23:31.938695 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:23:31.938695 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:23:31.938695 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:23:31.938695 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:23:31.938695 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:23:31.938695 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:23:31.938695 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:23:31.939799 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:23:31.939799 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:23:31.939799 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:23:31.939799 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:23:31.941248 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 613/1215] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:23:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:23:31.944835 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78e6453-8f36-467c-a7fd-34a016cd2ec7 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:32.016208 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:32.062835 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78e6453-8f36-467c-a7fd-34a016cd2ec7 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:32.102932 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf1fa9a-ee06-44af-b1ef-78db927078e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:32.104433 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf1fa9a-ee06-44af-b1ef-78db927078e3 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:32.104433 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf1fa9a-ee06-44af-b1ef-78db927078e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:32.104433 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf1fa9a-ee06-44af-b1ef-78db927078e3 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:32.104664 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf1fa9a-ee06-44af-b1ef-78db927078e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:23:32.104930 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf1fa9a-ee06-44af-b1ef-78db927078e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:32.106273 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf1fa9a-ee06-44af-b1ef-78db927078e3 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:32.119626 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 603/1216] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Aug 6 01:23:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:23:32.125517 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e942f5c-d721-4e7a-b7ad-6dd790ee314b None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:32.204741 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:32.244887 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e942f5c-d721-4e7a-b7ad-6dd790ee314b None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:32.286788 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526afcf2-6249-4760-ab35-42e7bf7cc254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:32.287493 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526afcf2-6249-4760-ab35-42e7bf7cc254 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:32.288386 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526afcf2-6249-4760-ab35-42e7bf7cc254 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:32.288688 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526afcf2-6249-4760-ab35-42e7bf7cc254 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:32.296814 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526afcf2-6249-4760-ab35-42e7bf7cc254 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:32.298406 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526afcf2-6249-4760-ab35-42e7bf7cc254 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:32.316119 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-526afcf2-6249-4760-ab35-42e7bf7cc254 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:23:32.316119 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:23:32.316119 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:23:32.316119 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:23:32.316119 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:23:32.316119 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:23:32.316119 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:23:32.316119 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:23:32.316119 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:23:32.316119 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:23:32.316119 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:23:32.316119 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:23:32.316119 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 01:23:32.316119 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 01:23:32.316119 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 01:23:32.317333 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 01:23:32.317333 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:23:32.317333 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:23:32.317333 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:23:32.317333 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:23:32.317333 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:23:32.317333 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:23:32.317333 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:23:32.317333 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:23:32.317333 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:23:32.317333 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:23:32.317333 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:23:32.317333 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:23:32.317333 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:23:32.317333 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:23:32.317333 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:23:32.317333 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:23:32.318468 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 01:23:32.318468 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 01:23:32.318468 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 01:23:32.318468 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 01:23:32.318468 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 01:23:32.318468 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 01:23:32.318468 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:23:32.318468 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:23:32.318468 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 614/1217] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Aug 6 01:23:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:23:32.323185 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf1fa9a-ee06-44af-b1ef-78db927078e3 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:32.435835 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf1fa9a-ee06-44af-b1ef-78db927078e3 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:32.465403 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:32.481289 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb651f2-4e1c-45b2-8d9b-a0d69cd19b41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:32.481920 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb651f2-4e1c-45b2-8d9b-a0d69cd19b41 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:32.482207 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb651f2-4e1c-45b2-8d9b-a0d69cd19b41 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:32.482479 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb651f2-4e1c-45b2-8d9b-a0d69cd19b41 None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:32.483240 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb651f2-4e1c-45b2-8d9b-a0d69cd19b41 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:23:32.483544 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb651f2-4e1c-45b2-8d9b-a0d69cd19b41 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:32.484985 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb651f2-4e1c-45b2-8d9b-a0d69cd19b41 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:32.497109 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 604/1218] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Aug 6 01:23:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:23:32.503224 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526afcf2-6249-4760-ab35-42e7bf7cc254 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:32.615409 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526afcf2-6249-4760-ab35-42e7bf7cc254 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:32.645213 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:32.652039 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5d850b-e3dd-4b23-925d-90b077d77743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:32.652670 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5d850b-e3dd-4b23-925d-90b077d77743 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:32.652956 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5d850b-e3dd-4b23-925d-90b077d77743 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:32.653262 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5d850b-e3dd-4b23-925d-90b077d77743 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:32.654049 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5d850b-e3dd-4b23-925d-90b077d77743 None admin] RBAC: Adding query filter params (role.id=c9ce798acd4b482f8f391c6af97eb7a4, scope.project.id=d9d68d310bf24449a5fe20d2dc264a58, user.id=fa3b93ddd9464a2cba724d67b56b477a) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:23:32.654365 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5d850b-e3dd-4b23-925d-90b077d77743 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:32.655796 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5d850b-e3dd-4b23-925d-90b077d77743 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:32.669312 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 615/1219] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Aug 6 01:23:32 2022] GET /identity/v3/role_assignments?user.id=fa3b93ddd9464a2cba724d67b56b477a&scope.project.id=d9d68d310bf24449a5fe20d2dc264a58&role.id=c9ce798acd4b482f8f391c6af97eb7a4 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:23:34.876484 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69635bbb-382b-4446-a675-41bb668d9561 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:34.877038 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69635bbb-382b-4446-a675-41bb668d9561 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:34.877180 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69635bbb-382b-4446-a675-41bb668d9561 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:34.878996 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 605/1220] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:23:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:23:34.885254 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f496bd3a-be58-47bd-9fc6-bf1e0cd87cbb None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:34.885780 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f496bd3a-be58-47bd-9fc6-bf1e0cd87cbb None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:34.885875 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f496bd3a-be58-47bd-9fc6-bf1e0cd87cbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:34.904352 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-f496bd3a-be58-47bd-9fc6-bf1e0cd87cbb None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:23:34.905584 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-f496bd3a-be58-47bd-9fc6-bf1e0cd87cbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:34.983632 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 616/1221] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:23:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:23:35.007418 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69635bbb-382b-4446-a675-41bb668d9561 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:35.126000 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69635bbb-382b-4446-a675-41bb668d9561 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:35.166529 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6131fabf-6466-4602-866f-a58059ebc95c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:35.167195 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6131fabf-6466-4602-866f-a58059ebc95c None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:35.167558 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6131fabf-6466-4602-866f-a58059ebc95c None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:35.167913 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6131fabf-6466-4602-866f-a58059ebc95c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:35.180306 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6131fabf-6466-4602-866f-a58059ebc95c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:23:35.185584 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-6131fabf-6466-4602-866f-a58059ebc95c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 06 01:23:35.185584 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:23:35.185584 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:23:35.185584 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:23:35.185584 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:23:35.185584 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:23:35.185584 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:23:35.185584 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:23:35.185584 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:23:35.185584 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:23:35.185584 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:23:35.185584 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:23:35.185584 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:23:35.185584 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:23:35.185584 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:23:35.185584 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:23:35.186817 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:23:35.186817 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:23:35.186817 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:23:35.186817 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:23:35.186817 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:23:35.186817 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:23:35.186817 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:23:35.186817 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:23:35.186817 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:23:35.186817 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:23:35.186817 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:23:35.186817 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:23:35.186817 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:23:35.186817 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:23:35.186817 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:23:35.186817 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:23:35.187863 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:23:35.187863 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:23:35.187863 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:23:35.187863 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:23:35.187863 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:23:35.187863 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:23:35.187863 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:23:35.187863 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:23:35.187863 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:23:35.187863 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:23:35.187863 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 06 01:23:35.187863 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:23:35.187863 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 606/1222] 10.222.0.26 () {56 vars in 1102 bytes} [Sat Aug 6 01:23:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:23:35.193258 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f496bd3a-be58-47bd-9fc6-bf1e0cd87cbb None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:35.300435 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f496bd3a-be58-47bd-9fc6-bf1e0cd87cbb None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:35.325987 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:35.336977 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8416f72f-8670-4be9-b7e5-a8d0996b30b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:35.337631 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8416f72f-8670-4be9-b7e5-a8d0996b30b9 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:35.337906 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8416f72f-8670-4be9-b7e5-a8d0996b30b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:35.338187 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8416f72f-8670-4be9-b7e5-a8d0996b30b9 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:35.338910 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8416f72f-8670-4be9-b7e5-a8d0996b30b9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:23:35.339208 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8416f72f-8670-4be9-b7e5-a8d0996b30b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:35.340602 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8416f72f-8670-4be9-b7e5-a8d0996b30b9 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:35.348719 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 617/1223] 10.222.0.26 () {56 vars in 1103 bytes} [Sat Aug 6 01:23:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:23:35.356219 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6131fabf-6466-4602-866f-a58059ebc95c None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:35.405852 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:35.464232 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6131fabf-6466-4602-866f-a58059ebc95c None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:35.502746 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78e4727-b093-4606-a1fa-eb64790fce10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:35.503391 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78e4727-b093-4606-a1fa-eb64790fce10 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:35.503678 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78e4727-b093-4606-a1fa-eb64790fce10 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:35.503960 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78e4727-b093-4606-a1fa-eb64790fce10 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:35.514952 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78e4727-b093-4606-a1fa-eb64790fce10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:35.516414 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78e4727-b093-4606-a1fa-eb64790fce10 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:35.531200 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-b78e4727-b093-4606-a1fa-eb64790fce10 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:23:35.531200 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:23:35.531200 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:23:35.531200 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:23:35.531200 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:23:35.531200 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:23:35.531200 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:23:35.531200 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:23:35.531200 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:23:35.531200 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:23:35.531200 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:23:35.531200 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:23:35.531200 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:23:35.531200 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:23:35.531200 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:23:35.532094 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:23:35.532094 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:23:35.532094 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:23:35.532094 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:23:35.532094 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:23:35.532094 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:23:35.532094 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:23:35.532094 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:23:35.532094 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:23:35.532094 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:23:35.532094 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:23:35.532094 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:23:35.532094 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:23:35.532094 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:23:35.532094 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:23:35.532094 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:23:35.532094 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:23:35.533120 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:23:35.533120 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:23:35.533120 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:23:35.533120 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:23:35.533120 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 607/1224] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:23:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:23:35.538975 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8416f72f-8670-4be9-b7e5-a8d0996b30b9 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:35.643390 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8416f72f-8670-4be9-b7e5-a8d0996b30b9 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:35.679110 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f278f7-0fb3-4aba-88c9-caeeb3db0ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:35.679738 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f278f7-0fb3-4aba-88c9-caeeb3db0ff8 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:35.680020 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f278f7-0fb3-4aba-88c9-caeeb3db0ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:35.680294 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f278f7-0fb3-4aba-88c9-caeeb3db0ff8 None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:35.681012 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f278f7-0fb3-4aba-88c9-caeeb3db0ff8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:23:35.681352 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f278f7-0fb3-4aba-88c9-caeeb3db0ff8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:35.682720 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f278f7-0fb3-4aba-88c9-caeeb3db0ff8 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:35.695231 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 618/1225] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Aug 6 01:23:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:23:35.701284 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78e4727-b093-4606-a1fa-eb64790fce10 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:35.798704 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:35.806184 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78e4727-b093-4606-a1fa-eb64790fce10 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:35.842887 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c997fc7-1848-4fc0-9dd7-5a5fd5806971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:35.843569 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c997fc7-1848-4fc0-9dd7-5a5fd5806971 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:35.843888 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c997fc7-1848-4fc0-9dd7-5a5fd5806971 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:35.844208 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c997fc7-1848-4fc0-9dd7-5a5fd5806971 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:35.853025 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c997fc7-1848-4fc0-9dd7-5a5fd5806971 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:35.854478 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c997fc7-1848-4fc0-9dd7-5a5fd5806971 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:35.871162 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-1c997fc7-1848-4fc0-9dd7-5a5fd5806971 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 06 01:23:35.871162 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:23:35.871162 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:23:35.871162 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:23:35.871162 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:23:35.871162 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:23:35.871162 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:23:35.871162 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:23:35.871162 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:23:35.871162 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:23:35.871162 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:23:35.871162 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:23:35.871162 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 01:23:35.871162 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 01:23:35.871162 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 01:23:35.871162 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 01:23:35.872174 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:23:35.872174 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:23:35.872174 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 01:23:35.872174 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:23:35.872174 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 01:23:35.872174 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:23:35.872174 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:23:35.872174 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:23:35.872174 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:23:35.872174 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:23:35.872174 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:23:35.872174 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:23:35.872174 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:23:35.872174 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:23:35.872174 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:23:35.872174 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:23:35.872174 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:23:35.872174 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:23:35.873156 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:23:35.873156 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:23:35.873156 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 01:23:35.873156 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 01:23:35.873156 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 01:23:35.873156 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 01:23:35.873156 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 01:23:35.873156 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 01:23:35.873156 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 06 01:23:35.873156 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:23:35.873156 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 608/1226] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:23:35 2022] GET /identity/v3/users/placement => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:23:35.878669 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f278f7-0fb3-4aba-88c9-caeeb3db0ff8 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:35.929236 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:35.988386 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f278f7-0fb3-4aba-88c9-caeeb3db0ff8 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:36.023119 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0931d687-330e-4d51-a889-3e66f20d3ff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:36.023734 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0931d687-330e-4d51-a889-3e66f20d3ff5 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:36.024011 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0931d687-330e-4d51-a889-3e66f20d3ff5 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:36.024285 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0931d687-330e-4d51-a889-3e66f20d3ff5 None admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:36.025095 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0931d687-330e-4d51-a889-3e66f20d3ff5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:23:36.025406 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0931d687-330e-4d51-a889-3e66f20d3ff5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:36.026833 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0931d687-330e-4d51-a889-3e66f20d3ff5 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:36.048413 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 619/1227] 10.222.0.26 () {56 vars in 1143 bytes} [Sat Aug 6 01:23:35 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:23:36.049675 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c997fc7-1848-4fc0-9dd7-5a5fd5806971 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:36.173965 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c997fc7-1848-4fc0-9dd7-5a5fd5806971 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:36.214210 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a18dd3b-9e23-407a-9175-7015f3fedf24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:36.214890 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a18dd3b-9e23-407a-9175-7015f3fedf24 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:36.215210 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a18dd3b-9e23-407a-9175-7015f3fedf24 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:36.215590 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a18dd3b-9e23-407a-9175-7015f3fedf24 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:36.228045 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a18dd3b-9e23-407a-9175-7015f3fedf24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:36.229517 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a18dd3b-9e23-407a-9175-7015f3fedf24 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:36.246827 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-8a18dd3b-9e23-407a-9175-7015f3fedf24 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:23:36.246827 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:23:36.246827 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:23:36.246827 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:23:36.246827 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:23:36.246827 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:23:36.246827 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:23:36.246827 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:23:36.246827 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:23:36.246827 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:23:36.246827 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:23:36.246827 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:23:36.246827 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:23:36.246827 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:23:36.246827 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:23:36.247981 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:23:36.247981 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:23:36.247981 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:23:36.247981 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:23:36.247981 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:23:36.247981 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:23:36.247981 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:23:36.247981 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:23:36.247981 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:23:36.247981 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:23:36.247981 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:23:36.247981 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:23:36.247981 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:23:36.247981 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:23:36.247981 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:23:36.247981 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:23:36.247981 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:23:36.249162 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:23:36.249162 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:23:36.249162 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:23:36.249162 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:23:36.249162 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 609/1228] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:23:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:23:36.254371 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0931d687-330e-4d51-a889-3e66f20d3ff5 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:36.373274 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0931d687-330e-4d51-a889-3e66f20d3ff5 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:36.376258 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:36.412158 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2ca816-046c-4b51-9731-169b3e085cc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:36.412893 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2ca816-046c-4b51-9731-169b3e085cc8 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:36.413423 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2ca816-046c-4b51-9731-169b3e085cc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:36.413728 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2ca816-046c-4b51-9731-169b3e085cc8 None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:36.414437 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2ca816-046c-4b51-9731-169b3e085cc8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:23:36.414746 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2ca816-046c-4b51-9731-169b3e085cc8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:36.416121 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2ca816-046c-4b51-9731-169b3e085cc8 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:36.427777 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 620/1229] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Aug 6 01:23:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:23:36.434510 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a18dd3b-9e23-407a-9175-7015f3fedf24 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:36.447748 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:36.555231 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a18dd3b-9e23-407a-9175-7015f3fedf24 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:36.598912 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c81cd5-c8b3-4f6d-a64e-b30c6ecadcf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:36.599564 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c81cd5-c8b3-4f6d-a64e-b30c6ecadcf4 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:36.599871 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c81cd5-c8b3-4f6d-a64e-b30c6ecadcf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:36.600681 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c81cd5-c8b3-4f6d-a64e-b30c6ecadcf4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:36.608334 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c81cd5-c8b3-4f6d-a64e-b30c6ecadcf4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:36.610254 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c81cd5-c8b3-4f6d-a64e-b30c6ecadcf4 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:36.624504 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-18c81cd5-c8b3-4f6d-a64e-b30c6ecadcf4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:23:36.624504 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:23:36.624504 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:23:36.624504 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:23:36.624504 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:23:36.624504 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:23:36.624504 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:23:36.624504 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:23:36.624504 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:23:36.624504 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:23:36.624504 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:23:36.624504 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:23:36.624504 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 01:23:36.624504 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 01:23:36.624504 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 01:23:36.625637 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 01:23:36.625637 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:23:36.625637 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:23:36.625637 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:23:36.625637 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:23:36.625637 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:23:36.625637 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:23:36.625637 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:23:36.625637 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:23:36.625637 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:23:36.625637 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:23:36.625637 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:23:36.625637 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:23:36.625637 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:23:36.625637 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:23:36.625637 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:23:36.625637 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:23:36.626557 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 01:23:36.626557 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 01:23:36.626557 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 01:23:36.626557 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 01:23:36.626557 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 01:23:36.626557 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 01:23:36.626557 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:23:36.626557 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:23:36.626557 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 610/1230] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Aug 6 01:23:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:23:36.631824 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2ca816-046c-4b51-9731-169b3e085cc8 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:36.754695 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2ca816-046c-4b51-9731-169b3e085cc8 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:36.797714 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878fd562-59de-4d7f-80f4-b9bb2b5f076b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:36.798340 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878fd562-59de-4d7f-80f4-b9bb2b5f076b None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:36.798620 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878fd562-59de-4d7f-80f4-b9bb2b5f076b None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:36.798894 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878fd562-59de-4d7f-80f4-b9bb2b5f076b None admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:36.799647 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878fd562-59de-4d7f-80f4-b9bb2b5f076b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:23:36.799957 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878fd562-59de-4d7f-80f4-b9bb2b5f076b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:36.801464 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878fd562-59de-4d7f-80f4-b9bb2b5f076b None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:36.814765 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 621/1231] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Aug 6 01:23:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:23:36.821323 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c81cd5-c8b3-4f6d-a64e-b30c6ecadcf4 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:36.897556 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:36.940392 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c81cd5-c8b3-4f6d-a64e-b30c6ecadcf4 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:36.978647 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac5ce14-ad1e-45e0-be98-fcf78e30dc59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:36.979343 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac5ce14-ad1e-45e0-be98-fcf78e30dc59 None admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:36.979632 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac5ce14-ad1e-45e0-be98-fcf78e30dc59 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:36.979910 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac5ce14-ad1e-45e0-be98-fcf78e30dc59 None admin] PATH_INFO: `/v3/projects/d9d68d310bf24449a5fe20d2dc264a58/users/fa3b93ddd9464a2cba724d67b56b477a/roles/c9ce798acd4b482f8f391c6af97eb7a4` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:36.995985 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac5ce14-ad1e-45e0-be98-fcf78e30dc59 None admin] RBAC: Authorizing `identity:create_grant(project_id=d9d68d310bf24449a5fe20d2dc264a58, user_id=fa3b93ddd9464a2cba724d67b56b477a, role_id=c9ce798acd4b482f8f391c6af97eb7a4)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:36.997468 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac5ce14-ad1e-45e0-be98-fcf78e30dc59 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:37.011316 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 611/1232] 10.222.0.26 () {58 vars in 1439 bytes} [Sat Aug 6 01:23:36 2022] PUT /identity/v3/projects/d9d68d310bf24449a5fe20d2dc264a58/users/fa3b93ddd9464a2cba724d67b56b477a/roles/c9ce798acd4b482f8f391c6af97eb7a4 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:23:39.091212 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc171b56-bc13-407c-8b23-9c64fe096de9 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:39.091736 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc171b56-bc13-407c-8b23-9c64fe096de9 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:39.091824 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc171b56-bc13-407c-8b23-9c64fe096de9 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:39.093173 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 622/1233] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:23:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:23:39.099257 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3842e7fb-3c30-42e7-9d76-c95928723bd5 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:39.099742 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3842e7fb-3c30-42e7-9d76-c95928723bd5 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:39.099974 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3842e7fb-3c30-42e7-9d76-c95928723bd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:39.120230 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-3842e7fb-3c30-42e7-9d76-c95928723bd5 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:23:39.121686 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-3842e7fb-3c30-42e7-9d76-c95928723bd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:39.194897 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 612/1234] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:23:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:23:39.220376 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc171b56-bc13-407c-8b23-9c64fe096de9 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:39.248936 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:39.330756 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc171b56-bc13-407c-8b23-9c64fe096de9 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:39.367926 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf5367b-66f2-4dae-8d44-b44aa09a9ac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:39.368486 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf5367b-66f2-4dae-8d44-b44aa09a9ac9 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:39.368765 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf5367b-66f2-4dae-8d44-b44aa09a9ac9 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:39.369033 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf5367b-66f2-4dae-8d44-b44aa09a9ac9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:39.379289 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf5367b-66f2-4dae-8d44-b44aa09a9ac9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:23:39.384903 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-aaf5367b-66f2-4dae-8d44-b44aa09a9ac9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 06 01:23:39.384903 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:23:39.384903 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:23:39.384903 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:23:39.384903 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:23:39.384903 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:23:39.384903 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:23:39.384903 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:23:39.384903 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:23:39.384903 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:23:39.384903 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:23:39.384903 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:23:39.384903 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:23:39.384903 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:23:39.384903 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:23:39.384903 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:23:39.386019 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:23:39.386019 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:23:39.386019 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:23:39.386019 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:23:39.386019 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:23:39.386019 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:23:39.386019 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:23:39.386019 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:23:39.386019 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:23:39.386019 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:23:39.386019 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:23:39.386019 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:23:39.386019 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:23:39.386019 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:23:39.386019 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:23:39.386019 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:23:39.387017 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:23:39.387017 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:23:39.387017 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:23:39.387017 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:23:39.387017 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:23:39.387017 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:23:39.387017 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:23:39.387017 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:23:39.387017 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:23:39.387017 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:23:39.387017 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 06 01:23:39.387017 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:23:39.387017 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 623/1235] 10.222.0.26 () {56 vars in 1102 bytes} [Sat Aug 6 01:23:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:23:39.392840 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3842e7fb-3c30-42e7-9d76-c95928723bd5 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:39.445620 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:39.500962 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3842e7fb-3c30-42e7-9d76-c95928723bd5 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:39.535867 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1451f3ff-5a8f-4e80-8bf3-13005c30b3ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:39.536492 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1451f3ff-5a8f-4e80-8bf3-13005c30b3ee None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:39.536778 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1451f3ff-5a8f-4e80-8bf3-13005c30b3ee None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:39.537077 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1451f3ff-5a8f-4e80-8bf3-13005c30b3ee None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:39.538007 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1451f3ff-5a8f-4e80-8bf3-13005c30b3ee None admin] RBAC: Adding query filter params (name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:23:39.538260 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1451f3ff-5a8f-4e80-8bf3-13005c30b3ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:39.539790 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1451f3ff-5a8f-4e80-8bf3-13005c30b3ee None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:39.548742 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 613/1236] 10.222.0.26 () {56 vars in 1103 bytes} [Sat Aug 6 01:23:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:23:39.555466 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf5367b-66f2-4dae-8d44-b44aa09a9ac9 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:39.650589 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:39.669006 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf5367b-66f2-4dae-8d44-b44aa09a9ac9 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:39.706502 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27909d4f-053e-430d-b59a-99c3fd8d6156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:39.707245 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27909d4f-053e-430d-b59a-99c3fd8d6156 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:39.707530 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27909d4f-053e-430d-b59a-99c3fd8d6156 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:39.707813 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27909d4f-053e-430d-b59a-99c3fd8d6156 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:39.719247 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27909d4f-053e-430d-b59a-99c3fd8d6156 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:39.720939 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27909d4f-053e-430d-b59a-99c3fd8d6156 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:39.738275 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-27909d4f-053e-430d-b59a-99c3fd8d6156 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:23:39.738275 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:23:39.738275 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:23:39.738275 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:23:39.738275 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:23:39.738275 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:23:39.738275 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:23:39.738275 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:23:39.738275 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:23:39.738275 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:23:39.738275 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:23:39.738275 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:23:39.738275 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:23:39.738275 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:23:39.738275 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:23:39.739639 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:23:39.739639 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:23:39.739639 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:23:39.739639 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:23:39.739639 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:23:39.739639 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:23:39.739639 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:23:39.739639 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:23:39.739639 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:23:39.739639 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:23:39.739639 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:23:39.739639 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:23:39.739639 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:23:39.739639 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:23:39.739639 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:23:39.739639 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:23:39.739639 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:23:39.741768 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:23:39.741768 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:23:39.741768 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:23:39.741768 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:23:39.741768 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 624/1237] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:23:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:23:39.745705 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1451f3ff-5a8f-4e80-8bf3-13005c30b3ee None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:39.856522 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1451f3ff-5a8f-4e80-8bf3-13005c30b3ee None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:39.866798 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:39.894196 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218d2553-6894-4e71-bb27-b9327ca979a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:39.894820 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218d2553-6894-4e71-bb27-b9327ca979a0 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:39.895129 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218d2553-6894-4e71-bb27-b9327ca979a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:39.895406 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218d2553-6894-4e71-bb27-b9327ca979a0 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:39.896242 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218d2553-6894-4e71-bb27-b9327ca979a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:23:39.896560 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218d2553-6894-4e71-bb27-b9327ca979a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:39.898175 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218d2553-6894-4e71-bb27-b9327ca979a0 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:39.913260 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 614/1238] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Aug 6 01:23:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:23:39.919215 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27909d4f-053e-430d-b59a-99c3fd8d6156 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:40.033537 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27909d4f-053e-430d-b59a-99c3fd8d6156 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:40.070571 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2990894b-84fc-427e-9a62-d1a5c7b6c88c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:40.071208 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2990894b-84fc-427e-9a62-d1a5c7b6c88c None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:40.071491 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2990894b-84fc-427e-9a62-d1a5c7b6c88c None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:40.071762 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2990894b-84fc-427e-9a62-d1a5c7b6c88c None admin] PATH_INFO: `/v3/users/placement` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:40.074468 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:40.079692 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2990894b-84fc-427e-9a62-d1a5c7b6c88c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:40.081323 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2990894b-84fc-427e-9a62-d1a5c7b6c88c None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:40.094368 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-2990894b-84fc-427e-9a62-d1a5c7b6c88c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 06 01:23:40.094368 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:23:40.094368 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:23:40.094368 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:23:40.094368 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:23:40.094368 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:23:40.094368 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:23:40.094368 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:23:40.094368 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:23:40.094368 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:23:40.094368 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:23:40.094368 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:23:40.094368 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 01:23:40.094368 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 01:23:40.094368 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 01:23:40.094368 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 01:23:40.095850 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:23:40.095850 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:23:40.095850 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 01:23:40.095850 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:23:40.095850 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 01:23:40.095850 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:23:40.095850 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:23:40.095850 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:23:40.095850 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:23:40.095850 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:23:40.095850 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:23:40.095850 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:23:40.095850 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:23:40.095850 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:23:40.095850 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:23:40.095850 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:23:40.095850 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:23:40.095850 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:23:40.096843 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:23:40.096843 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:23:40.096843 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 01:23:40.096843 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 01:23:40.096843 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 01:23:40.096843 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 01:23:40.096843 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 01:23:40.096843 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 01:23:40.096843 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 06 01:23:40.096843 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:23:40.096843 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 625/1239] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:23:39 2022] GET /identity/v3/users/placement => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:23:40.102670 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218d2553-6894-4e71-bb27-b9327ca979a0 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:40.218557 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218d2553-6894-4e71-bb27-b9327ca979a0 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:40.257994 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4401a5df-d914-4c64-afae-ea50cdde9cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:40.258747 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4401a5df-d914-4c64-afae-ea50cdde9cae None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:40.259085 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4401a5df-d914-4c64-afae-ea50cdde9cae None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:40.259398 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4401a5df-d914-4c64-afae-ea50cdde9cae None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:40.260370 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4401a5df-d914-4c64-afae-ea50cdde9cae None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:23:40.260844 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4401a5df-d914-4c64-afae-ea50cdde9cae None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:40.262720 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4401a5df-d914-4c64-afae-ea50cdde9cae None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:40.284432 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 615/1240] 10.222.0.26 () {56 vars in 1143 bytes} [Sat Aug 6 01:23:40 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:23:40.291399 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2990894b-84fc-427e-9a62-d1a5c7b6c88c None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:40.406284 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2990894b-84fc-427e-9a62-d1a5c7b6c88c None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:40.449162 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464327b3-4e55-4569-8bba-588a1fb2e039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:40.449793 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464327b3-4e55-4569-8bba-588a1fb2e039 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:40.450146 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464327b3-4e55-4569-8bba-588a1fb2e039 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:40.450498 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464327b3-4e55-4569-8bba-588a1fb2e039 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:40.520527 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:40.527924 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464327b3-4e55-4569-8bba-588a1fb2e039 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:40.529438 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464327b3-4e55-4569-8bba-588a1fb2e039 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:40.640978 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 626/1241] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Aug 6 01:23:40 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 06 01:23:40.647661 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4401a5df-d914-4c64-afae-ea50cdde9cae None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:40.676186 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:40.756731 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4401a5df-d914-4c64-afae-ea50cdde9cae None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:40.794019 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cf4282-7411-41c1-a59a-18bf660fb5e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:40.794606 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cf4282-7411-41c1-a59a-18bf660fb5e3 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:40.794899 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cf4282-7411-41c1-a59a-18bf660fb5e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:40.795228 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cf4282-7411-41c1-a59a-18bf660fb5e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:40.806481 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cf4282-7411-41c1-a59a-18bf660fb5e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:40.807983 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cf4282-7411-41c1-a59a-18bf660fb5e3 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:40.823896 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-91cf4282-7411-41c1-a59a-18bf660fb5e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:23:40.823896 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:23:40.823896 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:23:40.823896 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:23:40.823896 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:23:40.823896 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:23:40.823896 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:23:40.823896 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:23:40.823896 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:23:40.823896 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:23:40.823896 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:23:40.823896 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:23:40.823896 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:23:40.823896 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:23:40.823896 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:23:40.825034 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:23:40.825034 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:23:40.825034 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:23:40.825034 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:23:40.825034 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:23:40.825034 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:23:40.825034 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:23:40.825034 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:23:40.825034 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:23:40.825034 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:23:40.825034 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:23:40.825034 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:23:40.825034 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:23:40.825034 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:23:40.825034 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:23:40.825034 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:23:40.825034 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:23:40.831025 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:23:40.831025 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:23:40.831025 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:23:40.831025 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:23:40.831025 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 616/1242] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:23:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:23:40.831383 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464327b3-4e55-4569-8bba-588a1fb2e039 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:40.945129 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:40.952054 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464327b3-4e55-4569-8bba-588a1fb2e039 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:40.988133 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f626a4-ff09-4155-8822-1b2301c42db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:40.988831 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f626a4-ff09-4155-8822-1b2301c42db0 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:40.989174 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f626a4-ff09-4155-8822-1b2301c42db0 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:40.989456 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f626a4-ff09-4155-8822-1b2301c42db0 None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:40.990324 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f626a4-ff09-4155-8822-1b2301c42db0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:23:40.992083 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f626a4-ff09-4155-8822-1b2301c42db0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:40.993799 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f626a4-ff09-4155-8822-1b2301c42db0 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:41.008775 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 627/1243] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Aug 6 01:23:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:23:41.014551 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cf4282-7411-41c1-a59a-18bf660fb5e3 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:41.108392 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:41.136853 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cf4282-7411-41c1-a59a-18bf660fb5e3 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:41.176222 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2e2f27-c5cb-488f-b3a7-a88d609c27e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:41.176948 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2e2f27-c5cb-488f-b3a7-a88d609c27e5 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:41.177284 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2e2f27-c5cb-488f-b3a7-a88d609c27e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:41.177563 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2e2f27-c5cb-488f-b3a7-a88d609c27e5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:41.186237 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2e2f27-c5cb-488f-b3a7-a88d609c27e5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:41.187854 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2e2f27-c5cb-488f-b3a7-a88d609c27e5 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:41.199831 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-da2e2f27-c5cb-488f-b3a7-a88d609c27e5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:23:41.199831 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:23:41.199831 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:23:41.199831 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:23:41.199831 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:23:41.199831 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:23:41.199831 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:23:41.199831 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:23:41.199831 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:23:41.199831 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:23:41.199831 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:23:41.199831 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:23:41.199831 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 01:23:41.199831 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 01:23:41.199831 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 01:23:41.200879 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 01:23:41.200879 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:23:41.200879 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:23:41.200879 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:23:41.200879 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:23:41.200879 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:23:41.200879 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:23:41.200879 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:23:41.200879 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:23:41.200879 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:23:41.200879 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:23:41.200879 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:23:41.200879 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:23:41.200879 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:23:41.200879 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:23:41.200879 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:23:41.200879 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:23:41.202207 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 01:23:41.202207 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 01:23:41.202207 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 01:23:41.202207 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 01:23:41.202207 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 01:23:41.202207 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 01:23:41.202207 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:23:41.202207 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:23:41.205887 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 617/1244] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Aug 6 01:23:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:23:41.206886 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f626a4-ff09-4155-8822-1b2301c42db0 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:41.316893 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f626a4-ff09-4155-8822-1b2301c42db0 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:41.358590 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d4f111-6d92-406a-ae39-95fdcb990191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:41.359159 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d4f111-6d92-406a-ae39-95fdcb990191 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:41.359442 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d4f111-6d92-406a-ae39-95fdcb990191 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:41.359723 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d4f111-6d92-406a-ae39-95fdcb990191 None admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:41.360481 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d4f111-6d92-406a-ae39-95fdcb990191 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:23:41.360791 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d4f111-6d92-406a-ae39-95fdcb990191 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:41.362507 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d4f111-6d92-406a-ae39-95fdcb990191 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:41.377076 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 628/1245] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Aug 6 01:23:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:23:41.384106 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2e2f27-c5cb-488f-b3a7-a88d609c27e5 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:41.504219 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2e2f27-c5cb-488f-b3a7-a88d609c27e5 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:41.540112 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb2c8b7-8c3f-4904-81a7-14fe7d8f2e29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:41.540731 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb2c8b7-8c3f-4904-81a7-14fe7d8f2e29 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:41.541017 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb2c8b7-8c3f-4904-81a7-14fe7d8f2e29 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:41.541334 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb2c8b7-8c3f-4904-81a7-14fe7d8f2e29 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:41.542120 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb2c8b7-8c3f-4904-81a7-14fe7d8f2e29 None admin] RBAC: Adding query filter params (role.id=c9ce798acd4b482f8f391c6af97eb7a4, scope.project.id=d9d68d310bf24449a5fe20d2dc264a58, user.id=fa3b93ddd9464a2cba724d67b56b477a) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:23:41.542439 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb2c8b7-8c3f-4904-81a7-14fe7d8f2e29 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:41.543870 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb2c8b7-8c3f-4904-81a7-14fe7d8f2e29 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:41.550170 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 618/1246] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Aug 6 01:23:41 2022] GET /identity/v3/role_assignments?user.id=fa3b93ddd9464a2cba724d67b56b477a&scope.project.id=d9d68d310bf24449a5fe20d2dc264a58&role.id=c9ce798acd4b482f8f391c6af97eb7a4 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:23:43.632540 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3928786c-7497-41db-9961-65e9a7135fc4 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:43.633131 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3928786c-7497-41db-9961-65e9a7135fc4 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:43.633131 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3928786c-7497-41db-9961-65e9a7135fc4 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:43.634521 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 629/1247] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:23:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 01:23:43.639965 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28deb046-d79e-403a-8978-6dae0f69ef99 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:43.640397 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28deb046-d79e-403a-8978-6dae0f69ef99 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:43.640563 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28deb046-d79e-403a-8978-6dae0f69ef99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:43.648462 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:43.661563 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-28deb046-d79e-403a-8978-6dae0f69ef99 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:23:43.662990 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-28deb046-d79e-403a-8978-6dae0f69ef99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:43.737225 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 619/1248] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:23:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:23:43.764061 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3928786c-7497-41db-9961-65e9a7135fc4 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:43.785589 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:43.879226 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3928786c-7497-41db-9961-65e9a7135fc4 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:43.918847 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45b0fbc-ebed-4de7-b517-19f0595ce1a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:43.919799 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45b0fbc-ebed-4de7-b517-19f0595ce1a0 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:43.920285 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45b0fbc-ebed-4de7-b517-19f0595ce1a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:43.920763 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45b0fbc-ebed-4de7-b517-19f0595ce1a0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:43.937256 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45b0fbc-ebed-4de7-b517-19f0595ce1a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:23:43.943260 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-c45b0fbc-ebed-4de7-b517-19f0595ce1a0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:23:43.943260 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:23:43.943260 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:23:43.943260 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:23:43.943260 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:23:43.943260 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:23:43.943260 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:23:43.943260 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:23:43.943260 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:23:43.943260 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:23:43.943260 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:23:43.943260 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:23:43.943260 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:23:43.943260 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:23:43.943260 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:23:43.943260 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:23:43.944935 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:23:43.944935 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:23:43.944935 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:23:43.944935 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:23:43.944935 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:23:43.944935 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:23:43.944935 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:23:43.944935 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:23:43.944935 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:23:43.944935 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:23:43.944935 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:23:43.944935 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:23:43.944935 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:23:43.944935 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:23:43.944935 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:23:43.944935 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:23:43.945857 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:23:43.945857 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:23:43.945857 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:23:43.945857 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:23:43.945857 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:23:43.945857 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:23:43.945857 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:23:43.945857 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:23:43.945857 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:23:43.945857 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:23:43.945857 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:23:43.945857 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:23:43.948102 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 630/1249] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Aug 6 01:23:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:23:43.952188 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28deb046-d79e-403a-8978-6dae0f69ef99 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:44.062789 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28deb046-d79e-403a-8978-6dae0f69ef99 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:44.101441 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:44.109569 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3a20fb-d1db-47db-a1e9-2a1120e4becf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:44.110373 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3a20fb-d1db-47db-a1e9-2a1120e4becf None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:44.110555 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3a20fb-d1db-47db-a1e9-2a1120e4becf None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:44.110850 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3a20fb-d1db-47db-a1e9-2a1120e4becf None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:44.111843 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3a20fb-d1db-47db-a1e9-2a1120e4becf None admin] RBAC: Adding query filter params (name=admin) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:23:44.112234 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3a20fb-d1db-47db-a1e9-2a1120e4becf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:44.114228 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3a20fb-d1db-47db-a1e9-2a1120e4becf None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:44.124872 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 620/1250] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Aug 6 01:23:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:23:44.132300 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45b0fbc-ebed-4de7-b517-19f0595ce1a0 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:44.224464 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:44.254724 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45b0fbc-ebed-4de7-b517-19f0595ce1a0 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:44.292721 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9911c823-de7f-4f29-81a3-79cb5acd1f4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:44.293495 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9911c823-de7f-4f29-81a3-79cb5acd1f4c None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:44.293881 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9911c823-de7f-4f29-81a3-79cb5acd1f4c None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:44.294276 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9911c823-de7f-4f29-81a3-79cb5acd1f4c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:44.306148 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9911c823-de7f-4f29-81a3-79cb5acd1f4c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:44.307701 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9911c823-de7f-4f29-81a3-79cb5acd1f4c None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:44.324531 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-9911c823-de7f-4f29-81a3-79cb5acd1f4c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:23:44.324531 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:23:44.324531 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:23:44.324531 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:23:44.324531 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:23:44.324531 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:23:44.324531 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:23:44.324531 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:23:44.324531 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:23:44.324531 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:23:44.324531 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:23:44.324531 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:23:44.324531 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:23:44.324531 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:23:44.324531 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:23:44.325729 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:23:44.325729 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:23:44.325729 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:23:44.325729 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:23:44.325729 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:23:44.325729 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:23:44.325729 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:23:44.325729 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:23:44.325729 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:23:44.325729 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:23:44.325729 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:23:44.325729 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:23:44.325729 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:23:44.325729 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:23:44.325729 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:23:44.325729 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:23:44.325729 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:23:44.326810 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:23:44.326810 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:23:44.326810 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:23:44.326810 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:23:44.326810 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 631/1251] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:23:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:23:44.331983 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3a20fb-d1db-47db-a1e9-2a1120e4becf None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:44.449419 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3a20fb-d1db-47db-a1e9-2a1120e4becf None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:44.489805 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9056b510-37c2-407b-8ccc-09a5372f804e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:44.490800 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9056b510-37c2-407b-8ccc-09a5372f804e None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:44.491080 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9056b510-37c2-407b-8ccc-09a5372f804e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:44.491355 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9056b510-37c2-407b-8ccc-09a5372f804e None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:44.492138 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9056b510-37c2-407b-8ccc-09a5372f804e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:23:44.492584 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9056b510-37c2-407b-8ccc-09a5372f804e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:44.494261 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9056b510-37c2-407b-8ccc-09a5372f804e None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:44.509197 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 621/1252] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Aug 6 01:23:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:23:44.515512 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9911c823-de7f-4f29-81a3-79cb5acd1f4c None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:44.632646 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9911c823-de7f-4f29-81a3-79cb5acd1f4c None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:44.661850 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:44.671574 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc52bfa0-0cfa-4831-84f5-acfd54f6fbfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:44.672226 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc52bfa0-0cfa-4831-84f5-acfd54f6fbfd None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:44.672511 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc52bfa0-0cfa-4831-84f5-acfd54f6fbfd None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:44.672791 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc52bfa0-0cfa-4831-84f5-acfd54f6fbfd None admin] PATH_INFO: `/v3/users/placement` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:44.681885 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc52bfa0-0cfa-4831-84f5-acfd54f6fbfd None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:44.683327 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc52bfa0-0cfa-4831-84f5-acfd54f6fbfd None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:44.696768 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-dc52bfa0-0cfa-4831-84f5-acfd54f6fbfd None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 06 01:23:44.696768 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:23:44.696768 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:23:44.696768 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:23:44.696768 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:23:44.696768 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:23:44.696768 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:23:44.696768 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:23:44.696768 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:23:44.696768 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:23:44.696768 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:23:44.696768 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:23:44.696768 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 01:23:44.696768 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 01:23:44.696768 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 01:23:44.696768 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 01:23:44.697845 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:23:44.697845 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:23:44.697845 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 01:23:44.697845 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:23:44.697845 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 01:23:44.697845 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:23:44.697845 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:23:44.697845 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:23:44.697845 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:23:44.697845 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:23:44.697845 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:23:44.697845 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:23:44.697845 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:23:44.697845 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:23:44.697845 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:23:44.697845 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:23:44.697845 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:23:44.697845 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:23:44.698793 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:23:44.698793 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:23:44.698793 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 01:23:44.698793 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 01:23:44.698793 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 01:23:44.698793 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 01:23:44.698793 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 01:23:44.698793 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 01:23:44.698793 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 06 01:23:44.698793 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:23:44.698793 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 632/1253] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:23:44 2022] GET /identity/v3/users/placement => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:23:44.705124 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9056b510-37c2-407b-8ccc-09a5372f804e None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:44.773335 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:44.827534 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9056b510-37c2-407b-8ccc-09a5372f804e None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:44.867629 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9f36f8-d6d2-41fc-a838-b527119c7d49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:44.868254 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9f36f8-d6d2-41fc-a838-b527119c7d49 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:44.868545 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9f36f8-d6d2-41fc-a838-b527119c7d49 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:44.868895 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9f36f8-d6d2-41fc-a838-b527119c7d49 None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:44.869786 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9f36f8-d6d2-41fc-a838-b527119c7d49 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:23:44.870132 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9f36f8-d6d2-41fc-a838-b527119c7d49 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:44.871764 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9f36f8-d6d2-41fc-a838-b527119c7d49 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:44.893607 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 622/1254] 10.222.0.26 () {56 vars in 1143 bytes} [Sat Aug 6 01:23:44 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:23:44.900899 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc52bfa0-0cfa-4831-84f5-acfd54f6fbfd None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:45.019687 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc52bfa0-0cfa-4831-84f5-acfd54f6fbfd None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:45.060563 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5e4cd5-8bd8-4e59-b58c-1eae93743f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:45.061220 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5e4cd5-8bd8-4e59-b58c-1eae93743f61 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:45.061507 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5e4cd5-8bd8-4e59-b58c-1eae93743f61 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:45.061782 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5e4cd5-8bd8-4e59-b58c-1eae93743f61 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:45.133853 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:45.151443 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5e4cd5-8bd8-4e59-b58c-1eae93743f61 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:45.153384 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5e4cd5-8bd8-4e59-b58c-1eae93743f61 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:45.263717 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 633/1255] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Aug 6 01:23:44 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 06 01:23:45.269799 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9f36f8-d6d2-41fc-a838-b527119c7d49 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:45.387843 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9f36f8-d6d2-41fc-a838-b527119c7d49 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:45.406156 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:45.432651 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903716ec-e020-4f07-8c0e-75265415a275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:45.433326 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903716ec-e020-4f07-8c0e-75265415a275 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:45.433607 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903716ec-e020-4f07-8c0e-75265415a275 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:45.433876 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903716ec-e020-4f07-8c0e-75265415a275 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:45.444462 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903716ec-e020-4f07-8c0e-75265415a275 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:45.445921 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903716ec-e020-4f07-8c0e-75265415a275 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:45.460486 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-903716ec-e020-4f07-8c0e-75265415a275 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:23:45.460486 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:23:45.460486 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:23:45.460486 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:23:45.460486 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:23:45.460486 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:23:45.460486 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:23:45.460486 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:23:45.460486 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:23:45.460486 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:23:45.460486 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:23:45.460486 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:23:45.460486 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:23:45.460486 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:23:45.460486 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:23:45.461498 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:23:45.461498 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:23:45.461498 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:23:45.461498 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:23:45.461498 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:23:45.461498 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:23:45.461498 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:23:45.461498 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:23:45.461498 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:23:45.461498 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:23:45.461498 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:23:45.461498 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:23:45.461498 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:23:45.461498 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:23:45.461498 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:23:45.461498 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:23:45.461498 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:23:45.462396 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:23:45.462396 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:23:45.462396 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:23:45.462396 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:23:45.462396 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 623/1256] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:23:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:23:45.467521 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5e4cd5-8bd8-4e59-b58c-1eae93743f61 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:45.567100 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:45.580973 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5e4cd5-8bd8-4e59-b58c-1eae93743f61 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:45.620385 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2b8abc-5447-4629-abf6-fd014ec38def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:45.621002 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2b8abc-5447-4629-abf6-fd014ec38def None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:45.621327 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2b8abc-5447-4629-abf6-fd014ec38def None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:45.621596 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2b8abc-5447-4629-abf6-fd014ec38def None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:45.622290 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2b8abc-5447-4629-abf6-fd014ec38def None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:23:45.622593 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2b8abc-5447-4629-abf6-fd014ec38def None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:45.623957 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2b8abc-5447-4629-abf6-fd014ec38def None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:45.636235 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 634/1257] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Aug 6 01:23:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:23:45.641764 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903716ec-e020-4f07-8c0e-75265415a275 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:45.756540 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903716ec-e020-4f07-8c0e-75265415a275 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:45.796607 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a84d40-0337-4277-a9b1-c59af38bc75f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:45.797340 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a84d40-0337-4277-a9b1-c59af38bc75f None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:45.797683 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a84d40-0337-4277-a9b1-c59af38bc75f None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:45.797936 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a84d40-0337-4277-a9b1-c59af38bc75f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:45.806192 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a84d40-0337-4277-a9b1-c59af38bc75f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:45.807946 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a84d40-0337-4277-a9b1-c59af38bc75f None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:45.821385 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-58a84d40-0337-4277-a9b1-c59af38bc75f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:23:45.821385 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:23:45.821385 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:23:45.821385 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:23:45.821385 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:23:45.821385 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:23:45.821385 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:23:45.821385 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:23:45.821385 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:23:45.821385 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:23:45.821385 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:23:45.821385 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:23:45.821385 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 01:23:45.821385 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 01:23:45.821385 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 01:23:45.822383 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 01:23:45.822383 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:23:45.822383 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:23:45.822383 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:23:45.822383 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:23:45.822383 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:23:45.822383 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:23:45.822383 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:23:45.822383 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:23:45.822383 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:23:45.822383 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:23:45.822383 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:23:45.822383 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:23:45.822383 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:23:45.822383 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:23:45.822383 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:23:45.822383 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:23:45.823281 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 01:23:45.823281 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 01:23:45.823281 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 01:23:45.823281 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 01:23:45.823281 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 01:23:45.823281 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 01:23:45.823281 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:23:45.823281 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:23:45.823281 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 624/1258] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Aug 6 01:23:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:23:45.828743 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2b8abc-5447-4629-abf6-fd014ec38def None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:45.937255 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2b8abc-5447-4629-abf6-fd014ec38def None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:45.975360 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d627b70-9f13-40d9-a232-4b58ce301a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:45.976058 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d627b70-9f13-40d9-a232-4b58ce301a2d None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:45.976380 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d627b70-9f13-40d9-a232-4b58ce301a2d None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:45.977033 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d627b70-9f13-40d9-a232-4b58ce301a2d None admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:45.977940 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d627b70-9f13-40d9-a232-4b58ce301a2d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:23:45.978232 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d627b70-9f13-40d9-a232-4b58ce301a2d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:45.980095 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d627b70-9f13-40d9-a232-4b58ce301a2d None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:46.000565 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 635/1259] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Aug 6 01:23:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:23:46.001769 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a84d40-0337-4277-a9b1-c59af38bc75f None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:46.032430 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:46.122747 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a84d40-0337-4277-a9b1-c59af38bc75f None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:46.160657 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4aebf0-80ba-4262-9143-ecf55c8850df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:46.161355 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4aebf0-80ba-4262-9143-ecf55c8850df None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:46.161629 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4aebf0-80ba-4262-9143-ecf55c8850df None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:46.161898 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4aebf0-80ba-4262-9143-ecf55c8850df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:46.163375 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4aebf0-80ba-4262-9143-ecf55c8850df None admin] RBAC: Adding query filter params (role.id=4559b9a0837249f895a13c70096f8d38, scope.project.id=d9d68d310bf24449a5fe20d2dc264a58, user.id=fa3b93ddd9464a2cba724d67b56b477a) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:23:46.163947 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4aebf0-80ba-4262-9143-ecf55c8850df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:46.165488 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4aebf0-80ba-4262-9143-ecf55c8850df None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:46.172152 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 625/1260] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Aug 6 01:23:46 2022] GET /identity/v3/role_assignments?user.id=fa3b93ddd9464a2cba724d67b56b477a&scope.project.id=d9d68d310bf24449a5fe20d2dc264a58&role.id=4559b9a0837249f895a13c70096f8d38 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:23:48.182112 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03edffac-5ddb-481c-9fa4-86bccfc71332 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:48.182704 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03edffac-5ddb-481c-9fa4-86bccfc71332 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:48.182704 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03edffac-5ddb-481c-9fa4-86bccfc71332 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:48.184008 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 636/1261] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:23:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:23:48.189075 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d3fb0f-4313-4096-a332-2546ac8b76cc None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:48.189485 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d3fb0f-4313-4096-a332-2546ac8b76cc None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:48.189642 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d3fb0f-4313-4096-a332-2546ac8b76cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:48.207877 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-20d3fb0f-4313-4096-a332-2546ac8b76cc None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:23:48.209742 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-20d3fb0f-4313-4096-a332-2546ac8b76cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:48.274380 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 626/1262] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:23:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:23:48.296366 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03edffac-5ddb-481c-9fa4-86bccfc71332 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:48.307717 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:48.399144 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03edffac-5ddb-481c-9fa4-86bccfc71332 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:48.432472 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b584c4-5cf7-475f-9588-2deaa2f27c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:48.433039 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b584c4-5cf7-475f-9588-2deaa2f27c30 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:48.433351 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b584c4-5cf7-475f-9588-2deaa2f27c30 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:48.433627 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b584c4-5cf7-475f-9588-2deaa2f27c30 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:48.444116 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b584c4-5cf7-475f-9588-2deaa2f27c30 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:23:48.449359 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-55b584c4-5cf7-475f-9588-2deaa2f27c30 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:23:48.449359 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:23:48.449359 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:23:48.449359 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:23:48.449359 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:23:48.449359 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:23:48.449359 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:23:48.449359 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:23:48.449359 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:23:48.449359 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:23:48.449359 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:23:48.449359 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:23:48.449359 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:23:48.449359 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:23:48.449359 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:23:48.449359 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:23:48.450539 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:23:48.450539 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:23:48.450539 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:23:48.450539 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:23:48.450539 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:23:48.450539 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:23:48.450539 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:23:48.450539 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:23:48.450539 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:23:48.450539 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:23:48.450539 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:23:48.450539 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:23:48.450539 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:23:48.450539 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:23:48.450539 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:23:48.450539 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:23:48.451569 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:23:48.451569 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:23:48.451569 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:23:48.451569 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:23:48.451569 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:23:48.451569 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:23:48.451569 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:23:48.451569 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:23:48.451569 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:23:48.451569 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:23:48.451569 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:23:48.451569 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:23:48.451569 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 637/1263] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Aug 6 01:23:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:23:48.456480 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d3fb0f-4313-4096-a332-2546ac8b76cc None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:48.474362 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:48.553819 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d3fb0f-4313-4096-a332-2546ac8b76cc None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:48.586770 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e1e4a0-69aa-445e-91f5-379fecce0ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:48.587374 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e1e4a0-69aa-445e-91f5-379fecce0ddc None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:48.587650 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e1e4a0-69aa-445e-91f5-379fecce0ddc None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:48.587939 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e1e4a0-69aa-445e-91f5-379fecce0ddc None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:48.588624 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e1e4a0-69aa-445e-91f5-379fecce0ddc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:23:48.588941 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e1e4a0-69aa-445e-91f5-379fecce0ddc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:48.590313 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e1e4a0-69aa-445e-91f5-379fecce0ddc None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:48.598094 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 627/1264] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Aug 6 01:23:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:23:48.604820 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b584c4-5cf7-475f-9588-2deaa2f27c30 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:48.669357 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:48.702305 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b584c4-5cf7-475f-9588-2deaa2f27c30 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:48.734915 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaf5dc4-d29c-4f12-979b-4a780f1f810a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:48.735544 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaf5dc4-d29c-4f12-979b-4a780f1f810a None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:48.735825 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaf5dc4-d29c-4f12-979b-4a780f1f810a None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:48.736098 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaf5dc4-d29c-4f12-979b-4a780f1f810a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:48.746488 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aaf5dc4-d29c-4f12-979b-4a780f1f810a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:48.747925 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aaf5dc4-d29c-4f12-979b-4a780f1f810a None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:48.762367 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-1aaf5dc4-d29c-4f12-979b-4a780f1f810a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:23:48.762367 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:23:48.762367 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:23:48.762367 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:23:48.762367 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:23:48.762367 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:23:48.762367 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:23:48.762367 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:23:48.762367 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:23:48.762367 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:23:48.762367 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:23:48.762367 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:23:48.762367 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:23:48.762367 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:23:48.762367 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:23:48.763418 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:23:48.763418 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:23:48.763418 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:23:48.763418 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:23:48.763418 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:23:48.763418 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:23:48.763418 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:23:48.763418 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:23:48.763418 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:23:48.763418 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:23:48.763418 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:23:48.763418 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:23:48.763418 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:23:48.763418 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:23:48.763418 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:23:48.763418 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:23:48.763418 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:23:48.764688 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:23:48.764688 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:23:48.764688 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:23:48.764688 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:23:48.764688 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 638/1265] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:23:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:23:48.769288 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e1e4a0-69aa-445e-91f5-379fecce0ddc None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:48.846795 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:48.867200 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e1e4a0-69aa-445e-91f5-379fecce0ddc None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:48.899291 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba2d09d-a205-4946-88e6-5a7293322680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:48.899905 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba2d09d-a205-4946-88e6-5a7293322680 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:48.900190 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba2d09d-a205-4946-88e6-5a7293322680 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:48.900469 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba2d09d-a205-4946-88e6-5a7293322680 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:48.901187 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba2d09d-a205-4946-88e6-5a7293322680 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:23:48.901497 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba2d09d-a205-4946-88e6-5a7293322680 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:48.902856 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba2d09d-a205-4946-88e6-5a7293322680 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:48.914652 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 628/1266] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Aug 6 01:23:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:23:48.920319 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaf5dc4-d29c-4f12-979b-4a780f1f810a None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:49.016898 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaf5dc4-d29c-4f12-979b-4a780f1f810a None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:49.036508 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:49.050768 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42395b91-a9b2-4f0a-8765-87bcc8f63bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:49.051398 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42395b91-a9b2-4f0a-8765-87bcc8f63bec None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:49.051679 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42395b91-a9b2-4f0a-8765-87bcc8f63bec None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:49.051953 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42395b91-a9b2-4f0a-8765-87bcc8f63bec None admin] PATH_INFO: `/v3/users/placement` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:49.060074 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42395b91-a9b2-4f0a-8765-87bcc8f63bec None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:49.061552 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42395b91-a9b2-4f0a-8765-87bcc8f63bec None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:49.072591 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-42395b91-a9b2-4f0a-8765-87bcc8f63bec None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 06 01:23:49.072591 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:23:49.072591 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:23:49.072591 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:23:49.072591 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:23:49.072591 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:23:49.072591 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:23:49.072591 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:23:49.072591 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:23:49.072591 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:23:49.072591 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:23:49.072591 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:23:49.072591 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 01:23:49.072591 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 01:23:49.072591 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 01:23:49.072591 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 01:23:49.073742 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:23:49.073742 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:23:49.073742 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 01:23:49.073742 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:23:49.073742 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 01:23:49.073742 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:23:49.073742 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:23:49.073742 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:23:49.073742 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:23:49.073742 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:23:49.073742 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:23:49.073742 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:23:49.073742 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:23:49.073742 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:23:49.073742 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:23:49.073742 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:23:49.073742 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:23:49.073742 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:23:49.074694 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:23:49.074694 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:23:49.074694 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 01:23:49.074694 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 01:23:49.074694 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 01:23:49.074694 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 01:23:49.074694 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 01:23:49.074694 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 01:23:49.074694 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 06 01:23:49.074694 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:23:49.074694 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 639/1267] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:23:48 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:23:49.079636 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba2d09d-a205-4946-88e6-5a7293322680 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:49.190613 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba2d09d-a205-4946-88e6-5a7293322680 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:49.223186 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a94e86-0ea2-4eec-bc04-76a0f3822963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:49.223798 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a94e86-0ea2-4eec-bc04-76a0f3822963 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:49.224082 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a94e86-0ea2-4eec-bc04-76a0f3822963 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:49.224359 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a94e86-0ea2-4eec-bc04-76a0f3822963 None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:49.226015 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a94e86-0ea2-4eec-bc04-76a0f3822963 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:23:49.226328 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a94e86-0ea2-4eec-bc04-76a0f3822963 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:49.228002 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a94e86-0ea2-4eec-bc04-76a0f3822963 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:49.236251 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:49.245501 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 629/1268] 10.222.0.26 () {56 vars in 1143 bytes} [Sat Aug 6 01:23:49 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:23:49.251210 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42395b91-a9b2-4f0a-8765-87bcc8f63bec None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:49.347023 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42395b91-a9b2-4f0a-8765-87bcc8f63bec None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:49.379804 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accee9f5-3230-48e6-9c5d-ca05c7bf2428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:49.380434 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accee9f5-3230-48e6-9c5d-ca05c7bf2428 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:49.381226 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accee9f5-3230-48e6-9c5d-ca05c7bf2428 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:49.381511 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accee9f5-3230-48e6-9c5d-ca05c7bf2428 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:49.391752 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accee9f5-3230-48e6-9c5d-ca05c7bf2428 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:49.393229 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accee9f5-3230-48e6-9c5d-ca05c7bf2428 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:49.408079 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-accee9f5-3230-48e6-9c5d-ca05c7bf2428 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:23:49.408079 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:23:49.408079 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:23:49.408079 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:23:49.408079 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:23:49.408079 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:23:49.408079 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:23:49.408079 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:23:49.408079 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:23:49.408079 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:23:49.408079 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:23:49.408079 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:23:49.408079 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:23:49.408079 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:23:49.408079 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:23:49.409168 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:23:49.409168 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:23:49.409168 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:23:49.409168 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:23:49.409168 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:23:49.409168 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:23:49.409168 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:23:49.409168 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:23:49.409168 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:23:49.409168 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:23:49.409168 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:23:49.409168 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:23:49.409168 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:23:49.409168 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:23:49.409168 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:23:49.409168 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:23:49.409168 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:23:49.410459 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:23:49.410459 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:23:49.410459 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:23:49.410459 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:23:49.410459 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 640/1269] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:23:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:23:49.415027 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a94e86-0ea2-4eec-bc04-76a0f3822963 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:49.511372 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a94e86-0ea2-4eec-bc04-76a0f3822963 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:49.544563 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae184593-652e-45cf-bbfb-7e70a73081ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:49.545200 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae184593-652e-45cf-bbfb-7e70a73081ba None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:49.545483 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae184593-652e-45cf-bbfb-7e70a73081ba None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:49.545756 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae184593-652e-45cf-bbfb-7e70a73081ba None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:49.546433 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae184593-652e-45cf-bbfb-7e70a73081ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:23:49.546738 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae184593-652e-45cf-bbfb-7e70a73081ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:49.548092 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae184593-652e-45cf-bbfb-7e70a73081ba None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:49.558960 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 630/1270] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Aug 6 01:23:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:23:49.564509 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accee9f5-3230-48e6-9c5d-ca05c7bf2428 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:49.596168 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:49.664113 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accee9f5-3230-48e6-9c5d-ca05c7bf2428 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:49.696834 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db22d10a-40c5-436c-8fe5-2f7090b48dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:49.697479 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db22d10a-40c5-436c-8fe5-2f7090b48dc6 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:49.697759 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db22d10a-40c5-436c-8fe5-2f7090b48dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:49.698031 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db22d10a-40c5-436c-8fe5-2f7090b48dc6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:49.704324 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db22d10a-40c5-436c-8fe5-2f7090b48dc6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:49.705777 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db22d10a-40c5-436c-8fe5-2f7090b48dc6 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:49.717774 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-db22d10a-40c5-436c-8fe5-2f7090b48dc6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:23:49.717774 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:23:49.717774 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:23:49.717774 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:23:49.717774 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:23:49.717774 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:23:49.717774 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:23:49.717774 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:23:49.717774 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:23:49.717774 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:23:49.717774 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:23:49.717774 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:23:49.717774 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 01:23:49.717774 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 01:23:49.717774 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 01:23:49.718931 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 01:23:49.718931 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:23:49.718931 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:23:49.718931 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:23:49.718931 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:23:49.718931 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:23:49.718931 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:23:49.718931 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:23:49.718931 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:23:49.718931 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:23:49.718931 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:23:49.718931 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:23:49.718931 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:23:49.718931 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:23:49.718931 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:23:49.718931 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:23:49.718931 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:23:49.720036 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 01:23:49.720036 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 01:23:49.720036 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 01:23:49.720036 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 01:23:49.720036 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 01:23:49.720036 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 01:23:49.720036 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:23:49.720036 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:23:49.720036 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 641/1271] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Aug 6 01:23:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:23:49.724571 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae184593-652e-45cf-bbfb-7e70a73081ba None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:49.782351 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:49.822838 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae184593-652e-45cf-bbfb-7e70a73081ba None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:49.854706 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c20ad3-2d7b-453e-8010-2fc36f651e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:49.855316 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c20ad3-2d7b-453e-8010-2fc36f651e0f None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:49.855599 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c20ad3-2d7b-453e-8010-2fc36f651e0f None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:49.855877 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c20ad3-2d7b-453e-8010-2fc36f651e0f None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:49.856621 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c20ad3-2d7b-453e-8010-2fc36f651e0f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:23:49.856929 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c20ad3-2d7b-453e-8010-2fc36f651e0f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:49.858392 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c20ad3-2d7b-453e-8010-2fc36f651e0f None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:49.870428 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 631/1272] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Aug 6 01:23:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:23:49.876319 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db22d10a-40c5-436c-8fe5-2f7090b48dc6 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:49.972425 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:49.982420 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db22d10a-40c5-436c-8fe5-2f7090b48dc6 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:50.016130 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bb3512-d6a4-41c0-bdc7-aa827da074f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:50.016778 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bb3512-d6a4-41c0-bdc7-aa827da074f8 None admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:50.017069 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bb3512-d6a4-41c0-bdc7-aa827da074f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:50.017357 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bb3512-d6a4-41c0-bdc7-aa827da074f8 None admin] PATH_INFO: `/v3/projects/d9d68d310bf24449a5fe20d2dc264a58/users/fa3b93ddd9464a2cba724d67b56b477a/roles/4559b9a0837249f895a13c70096f8d38` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:50.019260 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bb3512-d6a4-41c0-bdc7-aa827da074f8 None admin] RBAC: Authorizing `identity:create_grant(project_id=d9d68d310bf24449a5fe20d2dc264a58, user_id=fa3b93ddd9464a2cba724d67b56b477a, role_id=4559b9a0837249f895a13c70096f8d38)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:50.020671 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bb3512-d6a4-41c0-bdc7-aa827da074f8 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:50.033331 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 642/1273] 10.222.0.26 () {58 vars in 1439 bytes} [Sat Aug 6 01:23:49 2022] PUT /identity/v3/projects/d9d68d310bf24449a5fe20d2dc264a58/users/fa3b93ddd9464a2cba724d67b56b477a/roles/4559b9a0837249f895a13c70096f8d38 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:23:52.325152 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862dfd16-5380-4661-a335-87cb941e9ddb None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:52.325679 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862dfd16-5380-4661-a335-87cb941e9ddb None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:52.325764 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862dfd16-5380-4661-a335-87cb941e9ddb None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:52.327073 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 632/1274] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:23:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:23:52.332158 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd391a8-e2bd-49d5-8d1a-87c7bf0e96bf None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:52.332585 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd391a8-e2bd-49d5-8d1a-87c7bf0e96bf None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:52.332711 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd391a8-e2bd-49d5-8d1a-87c7bf0e96bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:52.351136 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-afd391a8-e2bd-49d5-8d1a-87c7bf0e96bf None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:23:52.352673 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-afd391a8-e2bd-49d5-8d1a-87c7bf0e96bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:52.418219 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 643/1275] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:23:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:23:52.442013 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862dfd16-5380-4661-a335-87cb941e9ddb None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:52.540638 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862dfd16-5380-4661-a335-87cb941e9ddb None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:52.557871 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:52.574491 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16f5ba7-edee-41ab-a9e1-c4a0e8428aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:52.575124 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16f5ba7-edee-41ab-a9e1-c4a0e8428aef None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:52.575457 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16f5ba7-edee-41ab-a9e1-c4a0e8428aef None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:52.575681 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16f5ba7-edee-41ab-a9e1-c4a0e8428aef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:52.585466 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16f5ba7-edee-41ab-a9e1-c4a0e8428aef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:23:52.590336 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-e16f5ba7-edee-41ab-a9e1-c4a0e8428aef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:23:52.590336 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:23:52.590336 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:23:52.590336 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:23:52.590336 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:23:52.590336 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:23:52.590336 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:23:52.590336 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:23:52.590336 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:23:52.590336 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:23:52.590336 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:23:52.590336 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:23:52.590336 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:23:52.590336 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:23:52.590336 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:23:52.590336 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:23:52.591395 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:23:52.591395 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:23:52.591395 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:23:52.591395 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:23:52.591395 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:23:52.591395 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:23:52.591395 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:23:52.591395 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:23:52.591395 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:23:52.591395 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:23:52.591395 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:23:52.591395 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:23:52.591395 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:23:52.591395 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:23:52.591395 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:23:52.591395 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:23:52.592411 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:23:52.592411 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:23:52.592411 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:23:52.592411 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:23:52.592411 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:23:52.592411 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:23:52.592411 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:23:52.592411 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:23:52.592411 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:23:52.592411 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:23:52.592411 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:23:52.592411 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:23:52.592411 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 633/1276] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Aug 6 01:23:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:23:52.597492 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd391a8-e2bd-49d5-8d1a-87c7bf0e96bf None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:52.675605 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:52.694411 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd391a8-e2bd-49d5-8d1a-87c7bf0e96bf None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:52.729114 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a88d7a-385f-4759-8970-25955b60b271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:52.729719 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a88d7a-385f-4759-8970-25955b60b271 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:52.729997 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a88d7a-385f-4759-8970-25955b60b271 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:52.730265 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a88d7a-385f-4759-8970-25955b60b271 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:52.730980 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a88d7a-385f-4759-8970-25955b60b271 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:23:52.731281 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a88d7a-385f-4759-8970-25955b60b271 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:52.732623 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a88d7a-385f-4759-8970-25955b60b271 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:52.740298 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 644/1277] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Aug 6 01:23:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:23:52.745975 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16f5ba7-edee-41ab-a9e1-c4a0e8428aef None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:52.843256 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16f5ba7-edee-41ab-a9e1-c4a0e8428aef None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:52.877061 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755f5aaa-26e7-41c7-99f5-c3594a0e7d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:52.877694 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755f5aaa-26e7-41c7-99f5-c3594a0e7d54 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:52.877974 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755f5aaa-26e7-41c7-99f5-c3594a0e7d54 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:52.878248 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755f5aaa-26e7-41c7-99f5-c3594a0e7d54 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:52.888002 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755f5aaa-26e7-41c7-99f5-c3594a0e7d54 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:52.889454 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755f5aaa-26e7-41c7-99f5-c3594a0e7d54 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:52.904035 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-755f5aaa-26e7-41c7-99f5-c3594a0e7d54 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:23:52.904035 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:23:52.904035 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:23:52.904035 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:23:52.904035 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:23:52.904035 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:23:52.904035 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:23:52.904035 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:23:52.904035 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:23:52.904035 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:23:52.904035 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:23:52.904035 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:23:52.904035 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:23:52.904035 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:23:52.904035 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:23:52.904913 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:23:52.904913 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:23:52.904913 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:23:52.904913 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:23:52.904913 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:23:52.904913 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:23:52.904913 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:23:52.904913 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:23:52.904913 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:23:52.904913 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:23:52.904913 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:23:52.904913 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:23:52.904913 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:23:52.904913 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:23:52.904913 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:23:52.904913 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:23:52.904913 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:23:52.905912 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:23:52.905912 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:23:52.905912 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:23:52.905912 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:23:52.905912 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 634/1278] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:23:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:23:52.910844 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a88d7a-385f-4759-8970-25955b60b271 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:53.008291 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a88d7a-385f-4759-8970-25955b60b271 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:53.041154 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fd35cb-e5f2-436d-b453-b5fd5d0f5990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:53.041766 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fd35cb-e5f2-436d-b453-b5fd5d0f5990 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:53.042049 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fd35cb-e5f2-436d-b453-b5fd5d0f5990 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:53.042324 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fd35cb-e5f2-436d-b453-b5fd5d0f5990 None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:53.043007 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fd35cb-e5f2-436d-b453-b5fd5d0f5990 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:23:53.043310 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fd35cb-e5f2-436d-b453-b5fd5d0f5990 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:53.044667 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fd35cb-e5f2-436d-b453-b5fd5d0f5990 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:53.056111 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 645/1279] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Aug 6 01:23:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:23:53.061892 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755f5aaa-26e7-41c7-99f5-c3594a0e7d54 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:53.093822 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:53.164873 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755f5aaa-26e7-41c7-99f5-c3594a0e7d54 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:53.197185 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5406e5b4-3781-4226-9e43-e16525f5700b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:53.197816 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5406e5b4-3781-4226-9e43-e16525f5700b None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:53.198100 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5406e5b4-3781-4226-9e43-e16525f5700b None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:53.198377 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5406e5b4-3781-4226-9e43-e16525f5700b None admin] PATH_INFO: `/v3/users/placement` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:53.205462 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5406e5b4-3781-4226-9e43-e16525f5700b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:53.206876 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5406e5b4-3781-4226-9e43-e16525f5700b None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:53.218377 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-5406e5b4-3781-4226-9e43-e16525f5700b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 06 01:23:53.218377 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:23:53.218377 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:23:53.218377 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:23:53.218377 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:23:53.218377 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:23:53.218377 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:23:53.218377 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:23:53.218377 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:23:53.218377 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:23:53.218377 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:23:53.218377 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:23:53.218377 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 01:23:53.218377 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 01:23:53.218377 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 01:23:53.218377 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 01:23:53.219490 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:23:53.219490 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:23:53.219490 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 01:23:53.219490 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:23:53.219490 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 01:23:53.219490 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:23:53.219490 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:23:53.219490 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:23:53.219490 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:23:53.219490 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:23:53.219490 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:23:53.219490 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:23:53.219490 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:23:53.219490 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:23:53.219490 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:23:53.219490 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:23:53.219490 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:23:53.219490 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:23:53.220479 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:23:53.220479 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:23:53.220479 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 01:23:53.220479 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 01:23:53.220479 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 01:23:53.220479 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 01:23:53.220479 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 01:23:53.220479 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 01:23:53.220479 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 06 01:23:53.220479 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:23:53.220479 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 635/1280] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:23:53 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:23:53.225359 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fd35cb-e5f2-436d-b453-b5fd5d0f5990 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:53.229443 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:53.321421 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fd35cb-e5f2-436d-b453-b5fd5d0f5990 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:53.353972 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5e464d-37c9-4ab0-9643-1685cff1bb49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:53.354625 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5e464d-37c9-4ab0-9643-1685cff1bb49 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:53.354776 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5e464d-37c9-4ab0-9643-1685cff1bb49 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:53.355045 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5e464d-37c9-4ab0-9643-1685cff1bb49 None admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:53.355821 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5e464d-37c9-4ab0-9643-1685cff1bb49 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:23:53.356124 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5e464d-37c9-4ab0-9643-1685cff1bb49 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:53.357572 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5e464d-37c9-4ab0-9643-1685cff1bb49 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:53.373826 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 646/1281] 10.222.0.26 () {56 vars in 1143 bytes} [Sat Aug 6 01:23:53 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:23:53.379561 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5406e5b4-3781-4226-9e43-e16525f5700b None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:53.470029 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:53.480190 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5406e5b4-3781-4226-9e43-e16525f5700b None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:53.513611 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71ca3f5-3140-4b2a-bbb4-8f33d0b81f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:53.514217 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71ca3f5-3140-4b2a-bbb4-8f33d0b81f81 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:53.514498 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71ca3f5-3140-4b2a-bbb4-8f33d0b81f81 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:53.514768 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71ca3f5-3140-4b2a-bbb4-8f33d0b81f81 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:53.579440 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71ca3f5-3140-4b2a-bbb4-8f33d0b81f81 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:53.580859 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71ca3f5-3140-4b2a-bbb4-8f33d0b81f81 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:53.666079 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:53.676291 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 636/1282] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Aug 6 01:23:53 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 06 01:23:53.682808 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5e464d-37c9-4ab0-9643-1685cff1bb49 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:53.743277 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:53.780603 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5e464d-37c9-4ab0-9643-1685cff1bb49 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:53.813442 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca96382-1225-4e0b-8a80-0b10c22c5c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:53.814074 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca96382-1225-4e0b-8a80-0b10c22c5c4b None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:53.814355 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca96382-1225-4e0b-8a80-0b10c22c5c4b None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:53.814632 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca96382-1225-4e0b-8a80-0b10c22c5c4b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:53.825020 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca96382-1225-4e0b-8a80-0b10c22c5c4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:53.826495 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca96382-1225-4e0b-8a80-0b10c22c5c4b None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:53.840612 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-2ca96382-1225-4e0b-8a80-0b10c22c5c4b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:23:53.840612 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:23:53.840612 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:23:53.840612 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:23:53.840612 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:23:53.840612 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:23:53.840612 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:23:53.840612 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:23:53.840612 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:23:53.840612 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:23:53.840612 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:23:53.840612 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:23:53.840612 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:23:53.840612 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:23:53.840612 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:23:53.841684 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:23:53.841684 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:23:53.841684 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:23:53.841684 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:23:53.841684 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:23:53.841684 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:23:53.841684 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:23:53.841684 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:23:53.841684 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:23:53.841684 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:23:53.841684 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:23:53.841684 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:23:53.841684 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:23:53.841684 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:23:53.841684 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:23:53.841684 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:23:53.841684 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:23:53.842801 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:23:53.842801 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:23:53.842801 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:23:53.842801 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:23:53.842801 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 647/1283] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:23:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:23:53.847521 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71ca3f5-3140-4b2a-bbb4-8f33d0b81f81 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:53.943951 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71ca3f5-3140-4b2a-bbb4-8f33d0b81f81 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:53.977163 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298a1bb9-d426-44d1-82fc-60d4428f1cde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:53.977769 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298a1bb9-d426-44d1-82fc-60d4428f1cde None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:53.978055 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298a1bb9-d426-44d1-82fc-60d4428f1cde None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:53.978329 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298a1bb9-d426-44d1-82fc-60d4428f1cde None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:53.979008 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298a1bb9-d426-44d1-82fc-60d4428f1cde None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:23:53.979311 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298a1bb9-d426-44d1-82fc-60d4428f1cde None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:53.980658 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298a1bb9-d426-44d1-82fc-60d4428f1cde None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:53.992129 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 637/1284] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Aug 6 01:23:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:23:53.997667 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca96382-1225-4e0b-8a80-0b10c22c5c4b None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:54.095977 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca96382-1225-4e0b-8a80-0b10c22c5c4b None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:54.112216 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:54.132761 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39e8b3e-d760-4320-9063-e66e9d5382f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:54.133420 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39e8b3e-d760-4320-9063-e66e9d5382f4 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:54.133704 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39e8b3e-d760-4320-9063-e66e9d5382f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:54.133976 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39e8b3e-d760-4320-9063-e66e9d5382f4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:54.140398 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39e8b3e-d760-4320-9063-e66e9d5382f4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:54.141859 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39e8b3e-d760-4320-9063-e66e9d5382f4 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:54.153715 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-b39e8b3e-d760-4320-9063-e66e9d5382f4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:23:54.153715 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:23:54.153715 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:23:54.153715 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:23:54.153715 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:23:54.153715 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:23:54.153715 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:23:54.153715 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:23:54.153715 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:23:54.153715 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:23:54.153715 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:23:54.153715 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:23:54.153715 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 01:23:54.153715 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 01:23:54.153715 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 01:23:54.154746 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 01:23:54.154746 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:23:54.154746 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:23:54.154746 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:23:54.154746 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:23:54.154746 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:23:54.154746 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:23:54.154746 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:23:54.154746 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 06 01:23:54.154746 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:23:54.154746 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:23:54.154746 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:23:54.154746 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:23:54.154746 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:23:54.154746 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:23:54.154746 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:23:54.154746 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:23:54.155744 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 01:23:54.155744 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 01:23:54.155744 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 01:23:54.155744 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 01:23:54.155744 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 01:23:54.155744 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 01:23:54.155744 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:23:54.155744 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 06 01:23:54.155744 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 648/1285] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Aug 6 01:23:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:23:54.160737 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298a1bb9-d426-44d1-82fc-60d4428f1cde None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:54.210669 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:54.259706 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298a1bb9-d426-44d1-82fc-60d4428f1cde None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:54.292336 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec3b043-d187-46ed-93cc-61ce2becdcbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:54.292942 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec3b043-d187-46ed-93cc-61ce2becdcbe None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:54.293257 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec3b043-d187-46ed-93cc-61ce2becdcbe None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:54.293531 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec3b043-d187-46ed-93cc-61ce2becdcbe None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:54.294268 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec3b043-d187-46ed-93cc-61ce2becdcbe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:23:54.294572 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec3b043-d187-46ed-93cc-61ce2becdcbe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:54.295984 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec3b043-d187-46ed-93cc-61ce2becdcbe None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:54.308075 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 638/1286] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Aug 6 01:23:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:23:54.314079 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39e8b3e-d760-4320-9063-e66e9d5382f4 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:54.409580 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39e8b3e-d760-4320-9063-e66e9d5382f4 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:54.442035 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d30632-0385-4f55-afc2-03238a8b4384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:54.442641 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d30632-0385-4f55-afc2-03238a8b4384 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:54.442917 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d30632-0385-4f55-afc2-03238a8b4384 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:54.443227 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d30632-0385-4f55-afc2-03238a8b4384 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:54.443994 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d30632-0385-4f55-afc2-03238a8b4384 None admin] RBAC: Adding query filter params (role.id=4559b9a0837249f895a13c70096f8d38, scope.project.id=d9d68d310bf24449a5fe20d2dc264a58, user.id=fa3b93ddd9464a2cba724d67b56b477a) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:23:54.444295 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d30632-0385-4f55-afc2-03238a8b4384 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:54.445746 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d30632-0385-4f55-afc2-03238a8b4384 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:54.451310 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 649/1287] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Aug 6 01:23:54 2022] GET /identity/v3/role_assignments?user.id=fa3b93ddd9464a2cba724d67b56b477a&scope.project.id=d9d68d310bf24449a5fe20d2dc264a58&role.id=4559b9a0837249f895a13c70096f8d38 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:23:56.451384 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dbc29c-11f9-420d-9ad7-bcfc2af214c8 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:56.451995 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dbc29c-11f9-420d-9ad7-bcfc2af214c8 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:56.451995 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dbc29c-11f9-420d-9ad7-bcfc2af214c8 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:56.453323 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 639/1288] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:23:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:23:56.458347 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1be987-d8b5-4eb4-be01-c0308db6cc94 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:56.459369 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1be987-d8b5-4eb4-be01-c0308db6cc94 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:56.459602 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1be987-d8b5-4eb4-be01-c0308db6cc94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:56.477978 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-4f1be987-d8b5-4eb4-be01-c0308db6cc94 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:23:56.479174 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-4f1be987-d8b5-4eb4-be01-c0308db6cc94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:56.508783 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:56.544938 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 650/1289] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:23:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:23:56.567069 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dbc29c-11f9-420d-9ad7-bcfc2af214c8 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:56.666237 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dbc29c-11f9-420d-9ad7-bcfc2af214c8 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:56.676362 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:56.700383 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315f7652-7942-49a8-9e1a-9ce1ed6917a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:56.701013 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315f7652-7942-49a8-9e1a-9ce1ed6917a1 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:56.701330 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315f7652-7942-49a8-9e1a-9ce1ed6917a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:56.701609 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315f7652-7942-49a8-9e1a-9ce1ed6917a1 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:56.702232 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315f7652-7942-49a8-9e1a-9ce1ed6917a1 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:56.703579 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315f7652-7942-49a8-9e1a-9ce1ed6917a1 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:56.712307 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-315f7652-7942-49a8-9e1a-9ce1ed6917a1 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 06 01:23:56.712307 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:23:56.712307 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 01:23:56.712307 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 01:23:56.712307 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 01:23:56.712307 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 01:23:56.712307 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 01:23:56.712307 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 01:23:56.712307 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 06 01:23:56.712307 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:23:56.712307 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 01:23:56.712307 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:23:56.712307 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:23:56.712307 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:23:56.712307 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:23:56.712307 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:23:56.712307 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:23:56.713598 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:23:56.713598 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:23:56.713598 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:23:56.713598 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:23:56.713598 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:23:56.713598 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:23:56.713598 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 01:23:56.713598 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 01:23:56.713598 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 01:23:56.713598 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 01:23:56.713598 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:23:56.713598 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:23:56.713598 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:23:56.713598 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:23:56.713598 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:23:56.713598 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:23:56.714719 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:23:56.714719 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:23:56.714719 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:23:56.714719 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:23:56.714719 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:23:56.714719 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:23:56.714719 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:23:56.714719 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:23:56.714719 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:23:56.714719 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:23:56.714719 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 01:23:56.714719 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 01:23:56.714719 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 06 01:23:56.714719 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:23:56.714719 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 640/1290] 10.222.0.26 () {56 vars in 1117 bytes} [Sat Aug 6 01:23:56 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:23:56.719565 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1be987-d8b5-4eb4-be01-c0308db6cc94 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:56.815699 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1be987-d8b5-4eb4-be01-c0308db6cc94 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:56.848745 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dbfcf0-6e64-4dc7-80f1-17eb6bfa66d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:56.849993 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dbfcf0-6e64-4dc7-80f1-17eb6bfa66d6 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:56.850283 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dbfcf0-6e64-4dc7-80f1-17eb6bfa66d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:56.850562 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dbfcf0-6e64-4dc7-80f1-17eb6bfa66d6 None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:56.851244 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dbfcf0-6e64-4dc7-80f1-17eb6bfa66d6 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:23:56.851553 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dbfcf0-6e64-4dc7-80f1-17eb6bfa66d6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:56.853188 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dbfcf0-6e64-4dc7-80f1-17eb6bfa66d6 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:56.857860 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 651/1291] 10.222.0.26 () {56 vars in 1116 bytes} [Sat Aug 6 01:23:56 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:23:56.863510 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315f7652-7942-49a8-9e1a-9ce1ed6917a1 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:56.960271 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315f7652-7942-49a8-9e1a-9ce1ed6917a1 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:56.994153 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1938dd50-d0f2-4258-9f1f-216cfb057541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:56.994758 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1938dd50-d0f2-4258-9f1f-216cfb057541 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:56.995041 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1938dd50-d0f2-4258-9f1f-216cfb057541 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:56.995318 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1938dd50-d0f2-4258-9f1f-216cfb057541 None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:56.995987 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1938dd50-d0f2-4258-9f1f-216cfb057541 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:23:56.996294 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1938dd50-d0f2-4258-9f1f-216cfb057541 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:56.997678 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1938dd50-d0f2-4258-9f1f-216cfb057541 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:57.002124 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 641/1292] 10.222.0.26 () {56 vars in 1116 bytes} [Sat Aug 6 01:23:56 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:23:58.974961 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c40569e-4240-4d49-b5bf-44c9f05e8061 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:58.975527 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c40569e-4240-4d49-b5bf-44c9f05e8061 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:58.975527 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c40569e-4240-4d49-b5bf-44c9f05e8061 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:58.977133 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 652/1293] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:23:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:23:58.982279 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564a2108-dc2d-488b-b9bf-22715f166015 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:58.982663 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564a2108-dc2d-488b-b9bf-22715f166015 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:58.982844 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564a2108-dc2d-488b-b9bf-22715f166015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:59.000885 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-564a2108-dc2d-488b-b9bf-22715f166015 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:23:59.002102 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-564a2108-dc2d-488b-b9bf-22715f166015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:59.030892 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:59.067098 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 642/1294] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:23:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:23:59.091666 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c40569e-4240-4d49-b5bf-44c9f05e8061 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:59.130490 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:59.211303 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c40569e-4240-4d49-b5bf-44c9f05e8061 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:59.243670 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8df0c3e-dfce-48e4-99a4-cef91183bb26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:59.244219 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8df0c3e-dfce-48e4-99a4-cef91183bb26 None admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:59.244500 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8df0c3e-dfce-48e4-99a4-cef91183bb26 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:59.244774 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8df0c3e-dfce-48e4-99a4-cef91183bb26 None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:59.245482 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8df0c3e-dfce-48e4-99a4-cef91183bb26 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:59.246845 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8df0c3e-dfce-48e4-99a4-cef91183bb26 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:59.260948 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 653/1295] 10.222.0.26 () {60 vars in 1141 bytes} [Sat Aug 6 01:23:59 2022] POST /identity/v3/services => generated 246 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:24:01.228756 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553f2279-d12a-4515-9ec8-2756997966e1 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:01.229322 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553f2279-d12a-4515-9ec8-2756997966e1 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:01.229322 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553f2279-d12a-4515-9ec8-2756997966e1 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:01.230684 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 643/1296] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:24:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:24:01.235767 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9733c817-d84f-44cc-9713-f1de2b904e9a None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:01.236159 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9733c817-d84f-44cc-9713-f1de2b904e9a None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:01.236332 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9733c817-d84f-44cc-9713-f1de2b904e9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:01.254489 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-9733c817-d84f-44cc-9713-f1de2b904e9a None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:24:01.256401 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-9733c817-d84f-44cc-9713-f1de2b904e9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:01.326248 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 654/1297] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:24:01 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:24:01.348279 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553f2279-d12a-4515-9ec8-2756997966e1 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:01.446988 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553f2279-d12a-4515-9ec8-2756997966e1 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:01.479615 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c22e744-f2b7-4994-a5f4-df593c0a9b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:01.480181 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c22e744-f2b7-4994-a5f4-df593c0a9b62 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:01.480460 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c22e744-f2b7-4994-a5f4-df593c0a9b62 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:01.480732 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c22e744-f2b7-4994-a5f4-df593c0a9b62 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:01.481375 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c22e744-f2b7-4994-a5f4-df593c0a9b62 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:01.482722 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c22e744-f2b7-4994-a5f4-df593c0a9b62 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:01.491857 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-2c22e744-f2b7-4994-a5f4-df593c0a9b62 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 06 01:24:01.491857 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:24:01.491857 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 01:24:01.491857 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 01:24:01.491857 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 01:24:01.491857 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 01:24:01.491857 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 01:24:01.491857 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 01:24:01.491857 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 06 01:24:01.491857 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:24:01.491857 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 01:24:01.491857 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:24:01.491857 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:24:01.491857 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:24:01.491857 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:24:01.491857 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:24:01.491857 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:24:01.493025 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:24:01.493025 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:24:01.493025 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:24:01.493025 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:24:01.493025 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:24:01.493025 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:24:01.493025 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 01:24:01.493025 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 01:24:01.493025 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 01:24:01.493025 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 01:24:01.493025 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:24:01.493025 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:24:01.493025 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:24:01.493025 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:24:01.493025 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:24:01.493025 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:24:01.493904 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:24:01.493904 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:24:01.493904 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:24:01.493904 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:24:01.493904 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:24:01.493904 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:24:01.493904 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:24:01.493904 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:24:01.493904 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:24:01.493904 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:24:01.493904 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 01:24:01.493904 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 01:24:01.493904 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 06 01:24:01.493904 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:24:01.493904 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 644/1298] 10.222.0.26 () {56 vars in 1117 bytes} [Sat Aug 6 01:24:01 2022] GET /identity/v3/services/placement => generated 90 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:24:01.499049 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9733c817-d84f-44cc-9713-f1de2b904e9a None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:01.517118 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:24:01.596101 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9733c817-d84f-44cc-9713-f1de2b904e9a None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:01.629006 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43131f6-e2e1-4153-bb28-e7630517c42a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:01.629642 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43131f6-e2e1-4153-bb28-e7630517c42a None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:01.629925 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43131f6-e2e1-4153-bb28-e7630517c42a None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:01.630201 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43131f6-e2e1-4153-bb28-e7630517c42a None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:01.631595 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43131f6-e2e1-4153-bb28-e7630517c42a None admin] RBAC: Adding query filter params (name=placement) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:24:01.631898 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43131f6-e2e1-4153-bb28-e7630517c42a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:01.633278 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43131f6-e2e1-4153-bb28-e7630517c42a None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:01.638280 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 655/1299] 10.222.0.26 () {56 vars in 1116 bytes} [Sat Aug 6 01:24:01 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:24:01.644310 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c22e744-f2b7-4994-a5f4-df593c0a9b62 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:01.675568 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:24:01.742873 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c22e744-f2b7-4994-a5f4-df593c0a9b62 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:01.775159 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec63c00e-ae70-4dcd-b01b-5c0937762062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:01.775767 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec63c00e-ae70-4dcd-b01b-5c0937762062 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:01.776052 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec63c00e-ae70-4dcd-b01b-5c0937762062 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:01.776328 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec63c00e-ae70-4dcd-b01b-5c0937762062 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:01.777097 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec63c00e-ae70-4dcd-b01b-5c0937762062 None admin] RBAC: Adding query filter params (interface=public, service_id=7d0b0eea1ffd4e29afef85ad6a1cc500, region_id=RegionOne) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:24:01.777410 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec63c00e-ae70-4dcd-b01b-5c0937762062 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:01.778759 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec63c00e-ae70-4dcd-b01b-5c0937762062 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:01.783772 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 645/1300] 10.222.0.26 () {56 vars in 1251 bytes} [Sat Aug 6 01:24:01 2022] GET /identity/v3/endpoints?service_id=7d0b0eea1ffd4e29afef85ad6a1cc500&interface=public®ion_id=RegionOne => generated 194 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:24:01.789626 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43131f6-e2e1-4153-bb28-e7630517c42a None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:01.869388 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:24:01.887612 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43131f6-e2e1-4153-bb28-e7630517c42a None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:01.920109 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b70bee-f950-42ee-9abb-2ebb6c5dafb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:01.920718 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b70bee-f950-42ee-9abb-2ebb6c5dafb3 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:01.921000 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b70bee-f950-42ee-9abb-2ebb6c5dafb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:01.921303 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b70bee-f950-42ee-9abb-2ebb6c5dafb3 None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:01.921950 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b70bee-f950-42ee-9abb-2ebb6c5dafb3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:01.923320 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b70bee-f950-42ee-9abb-2ebb6c5dafb3 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:01.930671 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 656/1301] 10.222.0.26 () {56 vars in 1087 bytes} [Sat Aug 6 01:24:01 2022] GET /identity/v3/services => generated 2191 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:24:03.908713 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8022a84-907e-466d-88fb-ceb06edee837 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:03.909270 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8022a84-907e-466d-88fb-ceb06edee837 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:03.909386 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8022a84-907e-466d-88fb-ceb06edee837 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:03.910654 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 646/1302] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:24:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:24:03.916138 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a337780-e348-4224-9196-2a384fee7751 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:03.916583 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a337780-e348-4224-9196-2a384fee7751 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:03.916685 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a337780-e348-4224-9196-2a384fee7751 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:03.935009 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-1a337780-e348-4224-9196-2a384fee7751 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:24:03.936197 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-1a337780-e348-4224-9196-2a384fee7751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:04.001814 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 657/1303] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:24:03 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:24:04.024399 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8022a84-907e-466d-88fb-ceb06edee837 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:04.122580 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:24:04.129619 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8022a84-907e-466d-88fb-ceb06edee837 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:04.162577 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6935ce-8f22-4d91-83d0-e47cad1148d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:04.163289 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6935ce-8f22-4d91-83d0-e47cad1148d8 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:04.163569 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6935ce-8f22-4d91-83d0-e47cad1148d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:04.163840 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6935ce-8f22-4d91-83d0-e47cad1148d8 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:04.164460 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6935ce-8f22-4d91-83d0-e47cad1148d8 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:04.165833 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6935ce-8f22-4d91-83d0-e47cad1148d8 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:04.174499 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-2b6935ce-8f22-4d91-83d0-e47cad1148d8 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 06 01:24:04.174499 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:24:04.174499 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 01:24:04.174499 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 01:24:04.174499 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 01:24:04.174499 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 01:24:04.174499 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 01:24:04.174499 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 01:24:04.174499 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 06 01:24:04.174499 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:24:04.174499 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 01:24:04.174499 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:24:04.174499 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:24:04.174499 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:24:04.174499 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:24:04.174499 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:24:04.174499 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:24:04.175713 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:24:04.175713 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:24:04.175713 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:24:04.175713 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:24:04.175713 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:24:04.175713 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:24:04.175713 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 01:24:04.175713 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 01:24:04.175713 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 01:24:04.175713 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 01:24:04.175713 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:24:04.175713 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:24:04.175713 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:24:04.175713 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:24:04.175713 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:24:04.175713 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:24:04.176780 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:24:04.176780 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 06 01:24:04.176780 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:24:04.176780 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:24:04.176780 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:24:04.176780 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:24:04.176780 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:24:04.176780 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:24:04.176780 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:24:04.176780 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:24:04.176780 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 01:24:04.176780 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 01:24:04.176780 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 06 01:24:04.176780 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 06 01:24:04.176780 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 647/1304] 10.222.0.26 () {56 vars in 1117 bytes} [Sat Aug 6 01:24:04 2022] GET /identity/v3/services/placement => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:24:04.181689 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a337780-e348-4224-9196-2a384fee7751 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:04.255430 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:24:04.278617 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a337780-e348-4224-9196-2a384fee7751 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:04.311596 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9654c5-aca8-4e68-acd1-d631ae61f1ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:04.312211 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9654c5-aca8-4e68-acd1-d631ae61f1ec None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:04.312494 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9654c5-aca8-4e68-acd1-d631ae61f1ec None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:04.312770 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9654c5-aca8-4e68-acd1-d631ae61f1ec None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:04.313477 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9654c5-aca8-4e68-acd1-d631ae61f1ec None admin] RBAC: Adding query filter params (name=placement) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:24:04.313786 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9654c5-aca8-4e68-acd1-d631ae61f1ec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:04.315150 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9654c5-aca8-4e68-acd1-d631ae61f1ec None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:04.320345 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 658/1305] 10.222.0.26 () {56 vars in 1116 bytes} [Sat Aug 6 01:24:04 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:24:04.326083 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6935ce-8f22-4d91-83d0-e47cad1148d8 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:04.424890 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6935ce-8f22-4d91-83d0-e47cad1148d8 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:04.457988 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7467d49-93ec-4e61-a77a-5fb00a12b2fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:04.459173 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7467d49-93ec-4e61-a77a-5fb00a12b2fc None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:04.459450 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7467d49-93ec-4e61-a77a-5fb00a12b2fc None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:04.459717 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7467d49-93ec-4e61-a77a-5fb00a12b2fc None admin] PATH_INFO: `/v3/endpoints` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:04.460428 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7467d49-93ec-4e61-a77a-5fb00a12b2fc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:04.461809 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7467d49-93ec-4e61-a77a-5fb00a12b2fc None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:04.480726 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 648/1306] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:24:04 2022] POST /identity/v3/endpoints => generated 331 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:24:14.317294 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=30 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:24:14.477191 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:25:07.360367 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8522eaa6-feaa-4762-89f2-c911f995e5d8 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:25:07.361421 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8522eaa6-feaa-4762-89f2-c911f995e5d8 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:25:07.361928 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8522eaa6-feaa-4762-89f2-c911f995e5d8 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:25:07.369473 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358f3a94-de88-423f-bc21-e82b24d50edc None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:25:07.370322 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358f3a94-de88-423f-bc21-e82b24d50edc None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:25:07.370799 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358f3a94-de88-423f-bc21-e82b24d50edc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:25:07.375993 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 659/1307] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:25:07 2022] GET /identity => generated 270 bytes in 17 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:25:07.464527 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-358f3a94-de88-423f-bc21-e82b24d50edc None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:25:07.466150 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-358f3a94-de88-423f-bc21-e82b24d50edc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:25:07.605128 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 649/1308] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:25:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:25:07.667433 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a417f279-6631-4f69-9f10-22d5774f9191 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:25:07.668132 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a417f279-6631-4f69-9f10-22d5774f9191 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:25:07.668592 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a417f279-6631-4f69-9f10-22d5774f9191 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:25:07.671083 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 660/1309] 10.222.0.26 () {54 vars in 965 bytes} [Sat Aug 6 01:25:07 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:25:07.682138 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08685fde-600b-4ded-a9ea-8e0beb613981 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:25:07.682528 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08685fde-600b-4ded-a9ea-8e0beb613981 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:25:07.682705 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08685fde-600b-4ded-a9ea-8e0beb613981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:25:07.809211 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-08685fde-600b-4ded-a9ea-8e0beb613981 None None] MFA Rules not processed for user `4ac0e31c84b4445fa167bbe4954066c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:25:07.810531 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-08685fde-600b-4ded-a9ea-8e0beb613981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:25:07.834588 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:25:07.899533 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 650/1310] 10.222.0.26 () {58 vars in 1062 bytes} [Sat Aug 6 01:25:07 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:25:07.908144 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a417f279-6631-4f69-9f10-22d5774f9191 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:25:07.910089 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-a417f279-6631-4f69-9f10-22d5774f9191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:25:08.059581 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a417f279-6631-4f69-9f10-22d5774f9191 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:25:08.141179 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152b094e-39aa-449f-b926-9b19d781c51c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:25:08.141730 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152b094e-39aa-449f-b926-9b19d781c51c service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:25:08.142010 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152b094e-39aa-449f-b926-9b19d781c51c service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:25:08.142285 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152b094e-39aa-449f-b926-9b19d781c51c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:25:08.143882 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-152b094e-39aa-449f-b926-9b19d781c51c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:25:08.200764 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:25:08.259013 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152b094e-39aa-449f-b926-9b19d781c51c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:25:08.260465 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152b094e-39aa-449f-b926-9b19d781c51c service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:25:08.278975 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 661/1311] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:25:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 01:25:10.575159 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35effb8-f1a7-4a19-ac13-a749823752c2 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:25:10.575707 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35effb8-f1a7-4a19-ac13-a749823752c2 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:25:10.575707 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35effb8-f1a7-4a19-ac13-a749823752c2 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:25:10.577184 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 651/1312] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:25:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:25:10.582410 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd21c56-9097-4944-859a-ea3e62ffb270 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:25:10.582823 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd21c56-9097-4944-859a-ea3e62ffb270 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:25:10.583029 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd21c56-9097-4944-859a-ea3e62ffb270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:25:10.669813 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-bcd21c56-9097-4944-859a-ea3e62ffb270 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:25:10.670950 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-bcd21c56-9097-4944-859a-ea3e62ffb270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:25:10.781610 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 662/1313] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:25:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:25:10.821015 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35effb8-f1a7-4a19-ac13-a749823752c2 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:25:10.822635 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-c35effb8-f1a7-4a19-ac13-a749823752c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:25:10.932693 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35effb8-f1a7-4a19-ac13-a749823752c2 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:25:11.008461 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:25:11.011420 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f674b18-20a3-410a-b2d4-e736aa4d6022 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:25:11.012404 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f674b18-20a3-410a-b2d4-e736aa4d6022 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:25:11.012967 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f674b18-20a3-410a-b2d4-e736aa4d6022 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:25:11.013891 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f674b18-20a3-410a-b2d4-e736aa4d6022 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:25:11.016517 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-4f674b18-20a3-410a-b2d4-e736aa4d6022 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:25:11.127425 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f674b18-20a3-410a-b2d4-e736aa4d6022 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:25:11.128856 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f674b18-20a3-410a-b2d4-e736aa4d6022 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:25:11.148511 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 652/1314] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:25:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:25:13.470586 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650890d8-cfa4-462d-9f21-0f0a8ccc343b None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:25:13.471118 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650890d8-cfa4-462d-9f21-0f0a8ccc343b None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:25:13.471118 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650890d8-cfa4-462d-9f21-0f0a8ccc343b None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:25:13.472499 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 663/1315] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:25:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:25:13.477813 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd018ac4-7658-4220-8fe7-5669e48cc14d None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:25:13.478218 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd018ac4-7658-4220-8fe7-5669e48cc14d None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:25:13.478371 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd018ac4-7658-4220-8fe7-5669e48cc14d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:25:13.543277 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-bd018ac4-7658-4220-8fe7-5669e48cc14d None None] MFA Rules not processed for user `1cf42c5c30ba4b9a93fc4cdb9713c6cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:25:13.544471 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-bd018ac4-7658-4220-8fe7-5669e48cc14d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:25:13.596520 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:25:13.641591 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 653/1316] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:25:13 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:25:13.682242 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650890d8-cfa4-462d-9f21-0f0a8ccc343b None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:25:13.683804 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-650890d8-cfa4-462d-9f21-0f0a8ccc343b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:25:13.694353 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:25:13.785916 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650890d8-cfa4-462d-9f21-0f0a8ccc343b None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:25:13.858035 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4812220-c447-4f33-a55d-5fab96bc0fb2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:25:13.858656 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4812220-c447-4f33-a55d-5fab96bc0fb2 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:25:13.858939 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4812220-c447-4f33-a55d-5fab96bc0fb2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:25:13.859216 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4812220-c447-4f33-a55d-5fab96bc0fb2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:25:13.860960 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-c4812220-c447-4f33-a55d-5fab96bc0fb2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:25:13.922117 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:25:13.968538 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4812220-c447-4f33-a55d-5fab96bc0fb2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:25:13.970045 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4812220-c447-4f33-a55d-5fab96bc0fb2 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:25:13.990103 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 664/1317] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:25:13 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:25:16.563383 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b91951-f423-4a2e-b69c-ed30dc14e092 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:25:16.563927 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b91951-f423-4a2e-b69c-ed30dc14e092 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:25:16.564024 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b91951-f423-4a2e-b69c-ed30dc14e092 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:25:16.565374 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 654/1318] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:25:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:25:16.571090 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0a0626-c96f-4e5e-ae26-15dc58cf2511 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:25:16.571471 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0a0626-c96f-4e5e-ae26-15dc58cf2511 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:25:16.571628 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0a0626-c96f-4e5e-ae26-15dc58cf2511 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:25:16.636258 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-eb0a0626-c96f-4e5e-ae26-15dc58cf2511 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:25:16.637509 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-eb0a0626-c96f-4e5e-ae26-15dc58cf2511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:25:16.732522 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 665/1319] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:25:16 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:25:16.771080 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b91951-f423-4a2e-b69c-ed30dc14e092 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:25:16.772621 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-b7b91951-f423-4a2e-b69c-ed30dc14e092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:25:16.872625 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b91951-f423-4a2e-b69c-ed30dc14e092 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:25:16.943271 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9f5bba-cece-4a19-833d-482528f84cd5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:25:16.943877 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9f5bba-cece-4a19-833d-482528f84cd5 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:25:16.944155 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9f5bba-cece-4a19-833d-482528f84cd5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:25:16.944426 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9f5bba-cece-4a19-833d-482528f84cd5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:25:16.946059 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-3e9f5bba-cece-4a19-833d-482528f84cd5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:25:16.953165 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:25:17.052283 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9f5bba-cece-4a19-833d-482528f84cd5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:25:17.053737 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9f5bba-cece-4a19-833d-482528f84cd5 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:25:17.071827 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 655/1320] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:25:16 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:25:19.761183 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcad9883-e552-45f3-8e6f-27f8155e70fc None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:25:19.761750 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcad9883-e552-45f3-8e6f-27f8155e70fc None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:25:19.761750 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcad9883-e552-45f3-8e6f-27f8155e70fc None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:25:19.763063 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 666/1321] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:25:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:25:19.768290 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05b42ba-bc96-4e35-9771-a08ca41839f0 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:25:19.768678 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05b42ba-bc96-4e35-9771-a08ca41839f0 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:25:19.768975 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05b42ba-bc96-4e35-9771-a08ca41839f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:25:19.834901 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-c05b42ba-bc96-4e35-9771-a08ca41839f0 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:25:19.836126 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-c05b42ba-bc96-4e35-9771-a08ca41839f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:25:19.901108 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:25:19.945694 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 656/1322] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:25:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:25:19.981835 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcad9883-e552-45f3-8e6f-27f8155e70fc None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:25:19.983381 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-fcad9883-e552-45f3-8e6f-27f8155e70fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:25:20.004130 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:25:20.093913 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcad9883-e552-45f3-8e6f-27f8155e70fc None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:25:20.169933 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84f4ad5-e10d-410e-965c-a5726e237fe4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:25:20.170553 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84f4ad5-e10d-410e-965c-a5726e237fe4 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:25:20.170827 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84f4ad5-e10d-410e-965c-a5726e237fe4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:25:20.171096 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84f4ad5-e10d-410e-965c-a5726e237fe4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:25:20.172740 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-f84f4ad5-e10d-410e-965c-a5726e237fe4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:25:20.246248 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:25:20.284674 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84f4ad5-e10d-410e-965c-a5726e237fe4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:25:20.286578 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84f4ad5-e10d-410e-965c-a5726e237fe4 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:25:20.304977 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 667/1323] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:25:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:25:22.360649 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90526a50-61a4-4d4e-8de9-184f6c1a5c00 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:25:22.361181 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90526a50-61a4-4d4e-8de9-184f6c1a5c00 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:25:22.361181 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90526a50-61a4-4d4e-8de9-184f6c1a5c00 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:25:22.362568 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 657/1324] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:25:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:25:22.368180 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d068b6ca-f7cd-4809-ab9a-dc2faf38e610 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:25:22.368592 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d068b6ca-f7cd-4809-ab9a-dc2faf38e610 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:25:22.368739 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d068b6ca-f7cd-4809-ab9a-dc2faf38e610 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:25:22.436319 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-d068b6ca-f7cd-4809-ab9a-dc2faf38e610 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:25:22.437543 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-d068b6ca-f7cd-4809-ab9a-dc2faf38e610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:25:22.535081 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 668/1325] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:25:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:25:22.572211 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90526a50-61a4-4d4e-8de9-184f6c1a5c00 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:25:22.573944 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-90526a50-61a4-4d4e-8de9-184f6c1a5c00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:25:22.679315 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90526a50-61a4-4d4e-8de9-184f6c1a5c00 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:25:22.752425 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f3f75c-03e2-4ff6-aa61-781a8bbc273a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:25:22.753072 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f3f75c-03e2-4ff6-aa61-781a8bbc273a service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:25:22.753354 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f3f75c-03e2-4ff6-aa61-781a8bbc273a service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:25:22.753629 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f3f75c-03e2-4ff6-aa61-781a8bbc273a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:25:22.756105 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-69f3f75c-03e2-4ff6-aa61-781a8bbc273a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:25:22.776442 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:25:22.872965 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f3f75c-03e2-4ff6-aa61-781a8bbc273a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:25:22.874439 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f3f75c-03e2-4ff6-aa61-781a8bbc273a service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:25:22.893717 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 658/1326] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:25:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:25:24.958791 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91122033-1040-4793-a92f-4ba7567b83d7 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:25:24.959319 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91122033-1040-4793-a92f-4ba7567b83d7 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:25:24.959413 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91122033-1040-4793-a92f-4ba7567b83d7 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:25:24.960772 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 669/1327] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:25:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:25:24.966079 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dcf1fd-8acd-418a-9299-a1e96f78bbad None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:25:24.966465 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dcf1fd-8acd-418a-9299-a1e96f78bbad None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:25:24.966632 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dcf1fd-8acd-418a-9299-a1e96f78bbad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:25:25.033729 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-55dcf1fd-8acd-418a-9299-a1e96f78bbad None None] MFA Rules not processed for user `1cf42c5c30ba4b9a93fc4cdb9713c6cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:25:25.034951 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-55dcf1fd-8acd-418a-9299-a1e96f78bbad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:25:25.101027 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:25:25.134853 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 659/1328] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:25:24 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:25:25.172891 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91122033-1040-4793-a92f-4ba7567b83d7 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:25:25.174506 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-91122033-1040-4793-a92f-4ba7567b83d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:25:25.208276 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:25:25.285238 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91122033-1040-4793-a92f-4ba7567b83d7 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:25:25.358975 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b063f974-9dfd-4b64-bedc-cf496b0ab6b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:25:25.359602 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b063f974-9dfd-4b64-bedc-cf496b0ab6b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:25:25.359888 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b063f974-9dfd-4b64-bedc-cf496b0ab6b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:25:25.360169 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b063f974-9dfd-4b64-bedc-cf496b0ab6b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:25:25.361835 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-b063f974-9dfd-4b64-bedc-cf496b0ab6b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:25:25.446947 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:25:25.479385 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b063f974-9dfd-4b64-bedc-cf496b0ab6b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:25:25.480826 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b063f974-9dfd-4b64-bedc-cf496b0ab6b0 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:25:25.499730 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 670/1329] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:25:25 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:25:27.760857 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0191aa-3f0d-4ec1-8012-d5b755c73134 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:25:27.761394 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0191aa-3f0d-4ec1-8012-d5b755c73134 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:25:27.761394 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0191aa-3f0d-4ec1-8012-d5b755c73134 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:25:27.762749 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 660/1330] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:25:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:25:27.767895 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3126f405-e5ad-4af2-9432-13e7929a2cfe None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:25:27.768292 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3126f405-e5ad-4af2-9432-13e7929a2cfe None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:25:27.768447 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3126f405-e5ad-4af2-9432-13e7929a2cfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:25:27.834046 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-3126f405-e5ad-4af2-9432-13e7929a2cfe None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:25:27.835256 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-3126f405-e5ad-4af2-9432-13e7929a2cfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:25:27.931525 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 671/1331] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:25:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:25:27.967550 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0191aa-3f0d-4ec1-8012-d5b755c73134 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:25:27.969122 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-9c0191aa-3f0d-4ec1-8012-d5b755c73134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:25:28.071294 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0191aa-3f0d-4ec1-8012-d5b755c73134 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:25:28.144917 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f05c280-a1f7-4d3c-bd83-e5595e4973b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:25:28.145578 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f05c280-a1f7-4d3c-bd83-e5595e4973b5 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:25:28.145865 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f05c280-a1f7-4d3c-bd83-e5595e4973b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:25:28.146146 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f05c280-a1f7-4d3c-bd83-e5595e4973b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:25:28.147770 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-9f05c280-a1f7-4d3c-bd83-e5595e4973b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:25:28.173930 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:25:28.264109 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f05c280-a1f7-4d3c-bd83-e5595e4973b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:25:28.265569 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f05c280-a1f7-4d3c-bd83-e5595e4973b5 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:25:28.283413 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 661/1332] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:25:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:25:30.315907 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890ab350-6ee1-459b-8575-9e4c2ab3ba37 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:25:30.316503 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890ab350-6ee1-459b-8575-9e4c2ab3ba37 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:25:30.316503 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890ab350-6ee1-459b-8575-9e4c2ab3ba37 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:25:30.318483 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 672/1333] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:25:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:25:30.323648 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517b150a-6fe2-4665-8184-9e34788a2d1c None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:25:30.324067 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517b150a-6fe2-4665-8184-9e34788a2d1c None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:25:30.324207 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517b150a-6fe2-4665-8184-9e34788a2d1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:25:30.388410 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-517b150a-6fe2-4665-8184-9e34788a2d1c None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:25:30.389624 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-517b150a-6fe2-4665-8184-9e34788a2d1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:25:30.463926 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:25:30.486477 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 662/1334] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:25:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:25:30.542979 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf6ec03-8b29-4ffe-99e5-c7cc6c2f841a None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:25:30.543264 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf6ec03-8b29-4ffe-99e5-c7cc6c2f841a None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:25:30.543569 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf6ec03-8b29-4ffe-99e5-c7cc6c2f841a None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:25:30.544911 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 673/1335] 10.222.0.26 () {54 vars in 965 bytes} [Sat Aug 6 01:25:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:25:30.550763 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7a2ebe-88c0-471a-8b6c-597e0d95f85b None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:25:30.551648 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7a2ebe-88c0-471a-8b6c-597e0d95f85b None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:25:30.551960 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7a2ebe-88c0-471a-8b6c-597e0d95f85b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:25:30.627055 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-fb7a2ebe-88c0-471a-8b6c-597e0d95f85b None None] MFA Rules not processed for user `4ac0e31c84b4445fa167bbe4954066c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:25:30.628192 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-fb7a2ebe-88c0-471a-8b6c-597e0d95f85b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:25:30.702320 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 663/1336] 10.222.0.26 () {58 vars in 1062 bytes} [Sat Aug 6 01:25:30 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:25:30.711362 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf6ec03-8b29-4ffe-99e5-c7cc6c2f841a None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:25:30.712880 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-bbf6ec03-8b29-4ffe-99e5-c7cc6c2f841a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:25:30.752933 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:25:30.815320 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf6ec03-8b29-4ffe-99e5-c7cc6c2f841a None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:25:30.886231 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127b445d-e8ce-48b6-b464-e0f4e08081c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:25:30.886779 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127b445d-e8ce-48b6-b464-e0f4e08081c6 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:25:30.887053 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127b445d-e8ce-48b6-b464-e0f4e08081c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:25:30.887331 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127b445d-e8ce-48b6-b464-e0f4e08081c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:25:30.888929 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-127b445d-e8ce-48b6-b464-e0f4e08081c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:25:30.970032 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:25:31.002328 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127b445d-e8ce-48b6-b464-e0f4e08081c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:25:31.003761 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127b445d-e8ce-48b6-b464-e0f4e08081c6 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:25:31.023186 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 674/1337] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:25:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:25:33.899260 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bd692a-3ee1-4288-8af5-fae49e6dde6d None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:25:33.899853 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bd692a-3ee1-4288-8af5-fae49e6dde6d None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:25:33.899853 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bd692a-3ee1-4288-8af5-fae49e6dde6d None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:25:33.901318 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 664/1338] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:25:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:25:33.906497 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4aa109-508a-4aea-af86-d7b9eccd0f69 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:25:33.906902 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4aa109-508a-4aea-af86-d7b9eccd0f69 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:25:33.907054 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4aa109-508a-4aea-af86-d7b9eccd0f69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:25:33.971731 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-5a4aa109-508a-4aea-af86-d7b9eccd0f69 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:25:33.972932 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-5a4aa109-508a-4aea-af86-d7b9eccd0f69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:25:34.068009 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 675/1339] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:25:33 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:25:34.105608 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bd692a-3ee1-4288-8af5-fae49e6dde6d None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:25:34.107202 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-42bd692a-3ee1-4288-8af5-fae49e6dde6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:25:34.183139 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:25:34.211569 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bd692a-3ee1-4288-8af5-fae49e6dde6d None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:25:34.288400 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3a84b2-876a-47ba-a654-e440f09b142f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:25:34.289015 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3a84b2-876a-47ba-a654-e440f09b142f service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:25:34.289326 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3a84b2-876a-47ba-a654-e440f09b142f service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:25:34.289598 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3a84b2-876a-47ba-a654-e440f09b142f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:25:34.291217 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-0d3a84b2-876a-47ba-a654-e440f09b142f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:25:34.402210 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3a84b2-876a-47ba-a654-e440f09b142f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:25:34.403782 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3a84b2-876a-47ba-a654-e440f09b142f service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:25:34.423201 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 665/1340] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:25:34 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:25:38.482873 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76d85a0-0dab-40f8-a26f-dd0112c4d1ea None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:25:38.483456 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76d85a0-0dab-40f8-a26f-dd0112c4d1ea None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:25:38.483456 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76d85a0-0dab-40f8-a26f-dd0112c4d1ea None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:25:38.484782 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 676/1341] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:25:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:25:38.490220 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5917a6-861c-44dd-b4d1-7fb7376d925c None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:25:38.490644 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5917a6-861c-44dd-b4d1-7fb7376d925c None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:25:38.490789 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5917a6-861c-44dd-b4d1-7fb7376d925c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:25:38.496344 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:25:38.559453 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-ef5917a6-861c-44dd-b4d1-7fb7376d925c None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:25:38.560673 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-ef5917a6-861c-44dd-b4d1-7fb7376d925c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:25:38.661875 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 666/1342] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:25:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:25:38.701838 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76d85a0-0dab-40f8-a26f-dd0112c4d1ea None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:25:38.703466 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-f76d85a0-0dab-40f8-a26f-dd0112c4d1ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:25:38.753453 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:25:38.811389 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76d85a0-0dab-40f8-a26f-dd0112c4d1ea None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:25:38.885515 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351af805-6970-4b7d-918e-b040a2954d84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:25:38.886136 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351af805-6970-4b7d-918e-b040a2954d84 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:25:38.886415 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351af805-6970-4b7d-918e-b040a2954d84 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:25:38.886685 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351af805-6970-4b7d-918e-b040a2954d84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:25:38.888298 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-351af805-6970-4b7d-918e-b040a2954d84 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:25:38.987822 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:25:39.005505 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351af805-6970-4b7d-918e-b040a2954d84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:25:39.007051 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351af805-6970-4b7d-918e-b040a2954d84 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:25:39.036994 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 677/1343] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:25:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:25:41.656788 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92a4608-9b14-4b9b-a137-bc9de567d434 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:25:41.657371 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92a4608-9b14-4b9b-a137-bc9de567d434 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:25:41.657371 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92a4608-9b14-4b9b-a137-bc9de567d434 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:25:41.658664 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 667/1344] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:25:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:25:41.664821 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59162074-bab6-46c1-a817-902db33e2686 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:25:41.665356 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59162074-bab6-46c1-a817-902db33e2686 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:25:41.665581 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59162074-bab6-46c1-a817-902db33e2686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:25:41.737174 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-59162074-bab6-46c1-a817-902db33e2686 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:25:41.738402 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-59162074-bab6-46c1-a817-902db33e2686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:25:41.839387 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 678/1345] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:25:41 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:25:41.876966 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92a4608-9b14-4b9b-a137-bc9de567d434 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:25:41.878544 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-d92a4608-9b14-4b9b-a137-bc9de567d434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:25:41.965549 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:25:41.983351 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92a4608-9b14-4b9b-a137-bc9de567d434 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:25:42.058210 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60df53a6-d865-44a6-abe4-e7780c7e4401 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:25:42.058832 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60df53a6-d865-44a6-abe4-e7780c7e4401 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:25:42.059141 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60df53a6-d865-44a6-abe4-e7780c7e4401 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:25:42.059419 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60df53a6-d865-44a6-abe4-e7780c7e4401 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:25:42.061113 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-60df53a6-d865-44a6-abe4-e7780c7e4401 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:25:42.173416 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60df53a6-d865-44a6-abe4-e7780c7e4401 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:25:42.174858 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60df53a6-d865-44a6-abe4-e7780c7e4401 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:25:42.194016 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 668/1346] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:25:41 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:25:46.145469 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f45c703-197d-41cb-a48e-59bda28e6a70 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:25:46.146131 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f45c703-197d-41cb-a48e-59bda28e6a70 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:25:46.146131 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f45c703-197d-41cb-a48e-59bda28e6a70 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:25:46.147373 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 679/1347] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:25:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:25:46.152600 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3de421c-6f11-4748-920d-54e1a5c65af9 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:25:46.153020 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3de421c-6f11-4748-920d-54e1a5c65af9 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:25:46.153200 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3de421c-6f11-4748-920d-54e1a5c65af9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:25:46.170525 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:25:46.222061 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-f3de421c-6f11-4748-920d-54e1a5c65af9 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:25:46.223882 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-f3de421c-6f11-4748-920d-54e1a5c65af9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:25:46.326258 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 669/1348] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:25:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:25:46.364989 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f45c703-197d-41cb-a48e-59bda28e6a70 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:25:46.366634 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-2f45c703-197d-41cb-a48e-59bda28e6a70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:25:46.429768 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:25:46.482269 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f45c703-197d-41cb-a48e-59bda28e6a70 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:25:46.565017 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7499a377-ed3c-4543-b1b9-75bee718bf7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:25:46.565681 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7499a377-ed3c-4543-b1b9-75bee718bf7f service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:25:46.565957 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7499a377-ed3c-4543-b1b9-75bee718bf7f service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:25:46.566231 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7499a377-ed3c-4543-b1b9-75bee718bf7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:25:46.568641 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-7499a377-ed3c-4543-b1b9-75bee718bf7f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:25:46.685078 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:25:46.689485 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7499a377-ed3c-4543-b1b9-75bee718bf7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:25:46.690937 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7499a377-ed3c-4543-b1b9-75bee718bf7f service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:25:46.710794 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 680/1349] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:25:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:25:51.653789 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea381e3a-2395-4f0d-b8f7-6a6405f3a86c None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:25:51.654255 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea381e3a-2395-4f0d-b8f7-6a6405f3a86c None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:25:51.654430 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea381e3a-2395-4f0d-b8f7-6a6405f3a86c None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:25:51.655871 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 670/1350] 10.222.0.26 () {54 vars in 921 bytes} [Sat Aug 6 01:25:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:25:51.661269 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e437a752-f1e1-4c37-a4ef-a45745b2fee6 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:25:51.664747 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e437a752-f1e1-4c37-a4ef-a45745b2fee6 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:25:51.664747 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e437a752-f1e1-4c37-a4ef-a45745b2fee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:25:51.809633 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-e437a752-f1e1-4c37-a4ef-a45745b2fee6 None None] MFA Rules not processed for user `fa3b93ddd9464a2cba724d67b56b477a`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:25:51.811233 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-e437a752-f1e1-4c37-a4ef-a45745b2fee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:25:51.937020 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 681/1351] 10.222.0.26 () {58 vars in 1018 bytes} [Sat Aug 6 01:25:51 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:25:56.073878 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f9e0da-9699-4939-9150-943a6738388d None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:25:56.074419 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f9e0da-9699-4939-9150-943a6738388d None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:25:56.074419 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f9e0da-9699-4939-9150-943a6738388d None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:25:56.075975 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 671/1352] 10.222.0.26 () {54 vars in 921 bytes} [Sat Aug 6 01:25:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:25:56.080682 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef64f92-5bdb-44c1-b88a-53bfc389302e None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:25:56.081149 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef64f92-5bdb-44c1-b88a-53bfc389302e None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:25:56.081261 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef64f92-5bdb-44c1-b88a-53bfc389302e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:25:56.139605 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:25:56.167204 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-bef64f92-5bdb-44c1-b88a-53bfc389302e None None] MFA Rules not processed for user `fa3b93ddd9464a2cba724d67b56b477a`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:25:56.168420 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-bef64f92-5bdb-44c1-b88a-53bfc389302e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:25:56.292487 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 682/1353] 10.222.0.26 () {58 vars in 1018 bytes} [Sat Aug 6 01:25:56 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:25:56.324450 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=59 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:25:57.648655 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b0c23e-5d50-4b18-a9c1-83580fb39578 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:25:57.649268 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b0c23e-5d50-4b18-a9c1-83580fb39578 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:25:57.649268 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b0c23e-5d50-4b18-a9c1-83580fb39578 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:25:57.650522 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 672/1354] 10.222.0.26 () {54 vars in 921 bytes} [Sat Aug 6 01:25:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:25:57.655670 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035dd297-fc1d-43aa-b49a-d03b236c40f2 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:25:57.656077 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035dd297-fc1d-43aa-b49a-d03b236c40f2 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:25:57.656266 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035dd297-fc1d-43aa-b49a-d03b236c40f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:25:57.733950 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-035dd297-fc1d-43aa-b49a-d03b236c40f2 None None] MFA Rules not processed for user `fa3b93ddd9464a2cba724d67b56b477a`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:25:57.735174 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-035dd297-fc1d-43aa-b49a-d03b236c40f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:25:57.777133 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:25:57.842718 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 683/1355] 10.222.0.26 () {58 vars in 1018 bytes} [Sat Aug 6 01:25:57 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:26:01.274182 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2230a18a-e1ac-4381-b5a6-88ef547a397b None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:26:01.274747 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2230a18a-e1ac-4381-b5a6-88ef547a397b None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:26:01.274747 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2230a18a-e1ac-4381-b5a6-88ef547a397b None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:26:01.276270 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 673/1356] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:26:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:26:01.282822 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c990574b-16fd-4aa5-be48-4daecdb4aeda None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:26:01.283747 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c990574b-16fd-4aa5-be48-4daecdb4aeda None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:26:01.284342 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c990574b-16fd-4aa5-be48-4daecdb4aeda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:26:01.366092 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-c990574b-16fd-4aa5-be48-4daecdb4aeda None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:26:01.367508 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-c990574b-16fd-4aa5-be48-4daecdb4aeda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:26:01.478084 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 684/1357] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:26:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:26:01.536005 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee61801-7920-4ce8-a7e5-3e079f771abc None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:26:01.536851 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee61801-7920-4ce8-a7e5-3e079f771abc None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:26:01.537425 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee61801-7920-4ce8-a7e5-3e079f771abc None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:26:01.539403 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 674/1358] 10.222.0.26 () {54 vars in 962 bytes} [Sat Aug 6 01:26:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:26:01.547752 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f4eedd-7f2a-41f2-98cd-0cf2bc5f6e4f None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:26:01.548451 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f4eedd-7f2a-41f2-98cd-0cf2bc5f6e4f None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:26:01.548960 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f4eedd-7f2a-41f2-98cd-0cf2bc5f6e4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:26:01.568711 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:26:01.634875 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-26f4eedd-7f2a-41f2-98cd-0cf2bc5f6e4f None None] MFA Rules not processed for user `b87adc1d5bc240e99f95f6595b350be4`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:26:01.636118 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-26f4eedd-7f2a-41f2-98cd-0cf2bc5f6e4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:26:01.758437 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 685/1359] 10.222.0.26 () {58 vars in 1059 bytes} [Sat Aug 6 01:26:01 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:26:01.766057 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee61801-7920-4ce8-a7e5-3e079f771abc None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:26:01.767555 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-fee61801-7920-4ce8-a7e5-3e079f771abc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:26:01.924581 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee61801-7920-4ce8-a7e5-3e079f771abc None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:26:02.007290 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:26:02.052775 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1a1ebc-eae8-4ce2-8278-e5046fba6964 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87adc1d5bc240e99f95f6595b350be4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:26:02.053586 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1a1ebc-eae8-4ce2-8278-e5046fba6964 service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:26:02.053968 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1a1ebc-eae8-4ce2-8278-e5046fba6964 service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:26:02.054349 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1a1ebc-eae8-4ce2-8278-e5046fba6964 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:26:02.056060 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-6b1a1ebc-eae8-4ce2-8278-e5046fba6964 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:26:02.151292 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1a1ebc-eae8-4ce2-8278-e5046fba6964 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:26:02.153073 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1a1ebc-eae8-4ce2-8278-e5046fba6964 service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:26:02.181316 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 675/1360] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:26:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 01:26:04.406543 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bf7978-4bc8-4a6d-82f3-b04d4615a61a None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:26:04.407111 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bf7978-4bc8-4a6d-82f3-b04d4615a61a None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:26:04.407111 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bf7978-4bc8-4a6d-82f3-b04d4615a61a None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:26:04.409368 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 686/1361] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:26:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:26:04.414599 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c290312c-3886-4aee-a25c-0aa1b0c5c988 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:26:04.415126 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c290312c-3886-4aee-a25c-0aa1b0c5c988 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:26:04.415371 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c290312c-3886-4aee-a25c-0aa1b0c5c988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:26:04.487057 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-c290312c-3886-4aee-a25c-0aa1b0c5c988 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:26:04.488410 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-c290312c-3886-4aee-a25c-0aa1b0c5c988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:26:04.529574 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:26:04.601141 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 676/1362] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:26:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:26:04.641063 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bf7978-4bc8-4a6d-82f3-b04d4615a61a None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:26:04.643099 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-69bf7978-4bc8-4a6d-82f3-b04d4615a61a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:26:04.723271 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:26:04.821802 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bf7978-4bc8-4a6d-82f3-b04d4615a61a None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:26:04.964660 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac557203-0a94-4c26-acc7-304ec0bcee23 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87adc1d5bc240e99f95f6595b350be4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:26:04.965325 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac557203-0a94-4c26-acc7-304ec0bcee23 service nova] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:26:04.965613 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac557203-0a94-4c26-acc7-304ec0bcee23 service nova] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:26:04.965890 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac557203-0a94-4c26-acc7-304ec0bcee23 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:26:04.967577 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-ac557203-0a94-4c26-acc7-304ec0bcee23 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:26:04.993111 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:26:05.066398 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac557203-0a94-4c26-acc7-304ec0bcee23 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:26:05.068239 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac557203-0a94-4c26-acc7-304ec0bcee23 service nova] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:26:05.088278 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 687/1363] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:26:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:26:07.444052 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc77178-fba3-428c-a576-6b675c4ccc65 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:26:07.444603 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc77178-fba3-428c-a576-6b675c4ccc65 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:26:07.444603 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc77178-fba3-428c-a576-6b675c4ccc65 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:26:07.446038 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 677/1364] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:26:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:26:07.451528 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd5620e-16b4-4fd9-82d1-5850f02dd6de None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:26:07.452010 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd5620e-16b4-4fd9-82d1-5850f02dd6de None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:26:07.452111 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd5620e-16b4-4fd9-82d1-5850f02dd6de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:26:07.531616 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-3fd5620e-16b4-4fd9-82d1-5850f02dd6de None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:26:07.532847 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-3fd5620e-16b4-4fd9-82d1-5850f02dd6de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:26:07.655254 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:26:07.658460 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 688/1365] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:26:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:26:07.704185 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8326c90e-6592-404f-854c-bfaf3084521a None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:26:07.704663 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8326c90e-6592-404f-854c-bfaf3084521a None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:26:07.705093 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8326c90e-6592-404f-854c-bfaf3084521a None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:26:07.706780 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 678/1366] 10.222.0.26 () {54 vars in 962 bytes} [Sat Aug 6 01:26:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:26:07.713204 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0046ce-6025-4f98-9cc2-748a05e09895 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:26:07.713523 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0046ce-6025-4f98-9cc2-748a05e09895 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:26:07.713785 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0046ce-6025-4f98-9cc2-748a05e09895 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:26:07.795326 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-9a0046ce-6025-4f98-9cc2-748a05e09895 None None] MFA Rules not processed for user `b87adc1d5bc240e99f95f6595b350be4`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:26:07.796516 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-9a0046ce-6025-4f98-9cc2-748a05e09895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:26:07.920003 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 689/1367] 10.222.0.26 () {58 vars in 1059 bytes} [Sat Aug 6 01:26:07 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:26:07.927731 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8326c90e-6592-404f-854c-bfaf3084521a None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:26:07.929263 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-8326c90e-6592-404f-854c-bfaf3084521a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:26:08.092734 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8326c90e-6592-404f-854c-bfaf3084521a None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:26:08.117666 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:26:08.219977 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f53f05-9309-4af2-87dd-543c51b8a587 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87adc1d5bc240e99f95f6595b350be4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:26:08.220766 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f53f05-9309-4af2-87dd-543c51b8a587 service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:26:08.221180 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f53f05-9309-4af2-87dd-543c51b8a587 service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:26:08.221563 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f53f05-9309-4af2-87dd-543c51b8a587 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:26:08.223649 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-f2f53f05-9309-4af2-87dd-543c51b8a587 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:26:08.320891 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f53f05-9309-4af2-87dd-543c51b8a587 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:26:08.322693 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f53f05-9309-4af2-87dd-543c51b8a587 service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:26:08.342785 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 679/1368] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:26:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:26:09.463270 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0bcfc9-f52e-405f-960b-a5b08a80ae1a None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:26:09.463822 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0bcfc9-f52e-405f-960b-a5b08a80ae1a None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:26:09.463822 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0bcfc9-f52e-405f-960b-a5b08a80ae1a None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:26:09.465356 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 690/1369] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:26:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:26:09.470361 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2e6737-d742-4dc0-921d-c40264f9253b None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:26:09.470813 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2e6737-d742-4dc0-921d-c40264f9253b None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:26:09.470984 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2e6737-d742-4dc0-921d-c40264f9253b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:26:09.505547 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:26:09.547136 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-ce2e6737-d742-4dc0-921d-c40264f9253b None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:26:09.548376 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-ce2e6737-d742-4dc0-921d-c40264f9253b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:26:09.652880 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 680/1370] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:26:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:26:09.998587 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bfb96d-a81d-487e-bbc3-6c2cba553f2a None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:26:09.998587 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bfb96d-a81d-487e-bbc3-6c2cba553f2a None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:26:09.998587 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bfb96d-a81d-487e-bbc3-6c2cba553f2a None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:26:10.001342 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 691/1371] 10.222.0.26 () {54 vars in 964 bytes} [Sat Aug 6 01:26:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:26:10.005314 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c667e168-eb46-4f5f-8b86-14d80d867d33 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:26:10.005515 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c667e168-eb46-4f5f-8b86-14d80d867d33 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:26:10.005796 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c667e168-eb46-4f5f-8b86-14d80d867d33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:26:10.090150 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-c667e168-eb46-4f5f-8b86-14d80d867d33 None None] MFA Rules not processed for user `1be1e6173e374a60b9e1fbf51a1ab571`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:26:10.091398 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-c667e168-eb46-4f5f-8b86-14d80d867d33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:26:10.141474 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:26:10.174718 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 681/1372] 10.222.0.26 () {58 vars in 1061 bytes} [Sat Aug 6 01:26:10 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:26:10.182296 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bfb96d-a81d-487e-bbc3-6c2cba553f2a None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:26:10.183828 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-79bfb96d-a81d-487e-bbc3-6c2cba553f2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:26:10.240063 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:26:10.302596 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bfb96d-a81d-487e-bbc3-6c2cba553f2a None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:26:10.384225 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5007a403-3388-4f11-a9a8-9cad4307feed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1be1e6173e374a60b9e1fbf51a1ab571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:26:10.384979 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5007a403-3388-4f11-a9a8-9cad4307feed service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:26:10.385346 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5007a403-3388-4f11-a9a8-9cad4307feed service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:26:10.385631 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5007a403-3388-4f11-a9a8-9cad4307feed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:26:10.387655 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-5007a403-3388-4f11-a9a8-9cad4307feed service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:26:10.438380 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02415dc-695a-4066-a7bd-5cfa70f25abd None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:26:10.438892 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02415dc-695a-4066-a7bd-5cfa70f25abd None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:26:10.439404 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02415dc-695a-4066-a7bd-5cfa70f25abd None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:26:10.445092 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 682/1373] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:26:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:26:10.447777 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81f29c5-ace9-4e77-a832-150812239e5f None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:26:10.448128 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81f29c5-ace9-4e77-a832-150812239e5f None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:26:10.448706 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81f29c5-ace9-4e77-a832-150812239e5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:26:10.502472 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:26:10.521007 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5007a403-3388-4f11-a9a8-9cad4307feed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:26:10.522498 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5007a403-3388-4f11-a9a8-9cad4307feed service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:26:10.527690 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-a81f29c5-ace9-4e77-a832-150812239e5f None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:26:10.529063 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-a81f29c5-ace9-4e77-a832-150812239e5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:26:10.541991 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 692/1374] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:26:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:26:10.661970 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 683/1375] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:26:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:26:10.705887 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5007a403-3388-4f11-a9a8-9cad4307feed service cinder] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:26:10.707316 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-5007a403-3388-4f11-a9a8-9cad4307feed service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:26:10.807699 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5007a403-3388-4f11-a9a8-9cad4307feed service cinder] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:26:10.920609 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:26:10.933734 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1395a9a-28e7-43c3-bd8d-da7058804adf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87adc1d5bc240e99f95f6595b350be4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:26:10.934363 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1395a9a-28e7-43c3-bd8d-da7058804adf service nova] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:26:10.934672 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1395a9a-28e7-43c3-bd8d-da7058804adf service nova] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:26:10.934950 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1395a9a-28e7-43c3-bd8d-da7058804adf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:26:10.936604 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-c1395a9a-28e7-43c3-bd8d-da7058804adf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:26:11.026439 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1395a9a-28e7-43c3-bd8d-da7058804adf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:26:11.027956 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1395a9a-28e7-43c3-bd8d-da7058804adf service nova] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:26:11.046258 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 693/1376] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:26:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:26:13.087991 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b014013-a175-41ac-83c0-a6e4f489c8a9 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:26:13.088431 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b014013-a175-41ac-83c0-a6e4f489c8a9 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:26:13.088537 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b014013-a175-41ac-83c0-a6e4f489c8a9 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:26:13.089967 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 684/1377] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:26:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:26:13.095289 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9b6ce9-b4b6-45ce-bde0-b08a86a9d3c2 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:26:13.095729 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9b6ce9-b4b6-45ce-bde0-b08a86a9d3c2 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:26:13.096000 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9b6ce9-b4b6-45ce-bde0-b08a86a9d3c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:26:13.168057 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-3e9b6ce9-b4b6-45ce-bde0-b08a86a9d3c2 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:26:13.168380 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-3e9b6ce9-b4b6-45ce-bde0-b08a86a9d3c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:26:13.254349 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:26:13.294507 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 694/1378] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:26:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:26:13.338528 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b014013-a175-41ac-83c0-a6e4f489c8a9 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:26:13.341242 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-5b014013-a175-41ac-83c0-a6e4f489c8a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:26:13.396800 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:26:13.494826 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b014013-a175-41ac-83c0-a6e4f489c8a9 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:26:13.635087 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b42590f-24b1-40f2-bf78-e58898b4540e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87adc1d5bc240e99f95f6595b350be4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:26:13.636131 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b42590f-24b1-40f2-bf78-e58898b4540e service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:26:13.636575 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b42590f-24b1-40f2-bf78-e58898b4540e service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:26:13.636999 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b42590f-24b1-40f2-bf78-e58898b4540e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:26:13.638944 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-8b42590f-24b1-40f2-bf78-e58898b4540e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:26:13.652495 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:26:13.733847 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b42590f-24b1-40f2-bf78-e58898b4540e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:26:13.735456 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b42590f-24b1-40f2-bf78-e58898b4540e service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:26:13.753405 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 685/1379] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:26:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:26:15.815493 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a5fffd-f206-4838-ac88-319bb909a592 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:26:15.816035 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a5fffd-f206-4838-ac88-319bb909a592 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:26:15.816035 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a5fffd-f206-4838-ac88-319bb909a592 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:26:15.818125 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 695/1380] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:26:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:26:15.823410 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919dad6f-a8cf-4d92-b749-18250397c604 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:26:15.823847 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919dad6f-a8cf-4d92-b749-18250397c604 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:26:15.823975 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919dad6f-a8cf-4d92-b749-18250397c604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:26:15.892609 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-919dad6f-a8cf-4d92-b749-18250397c604 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:26:15.893848 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-919dad6f-a8cf-4d92-b749-18250397c604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:26:15.989233 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:26:16.003579 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 686/1381] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:26:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:26:16.048243 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a5fffd-f206-4838-ac88-319bb909a592 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:26:16.049995 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-02a5fffd-f206-4838-ac88-319bb909a592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:26:16.216048 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a5fffd-f206-4838-ac88-319bb909a592 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:26:16.267575 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:26:16.339290 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98727d64-4e79-480c-b8c4-575768fc93f5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87adc1d5bc240e99f95f6595b350be4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:26:16.339913 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98727d64-4e79-480c-b8c4-575768fc93f5 service nova] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:26:16.340201 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98727d64-4e79-480c-b8c4-575768fc93f5 service nova] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:26:16.340475 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98727d64-4e79-480c-b8c4-575768fc93f5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:26:16.342112 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-98727d64-4e79-480c-b8c4-575768fc93f5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:26:16.452906 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98727d64-4e79-480c-b8c4-575768fc93f5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:26:16.454702 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98727d64-4e79-480c-b8c4-575768fc93f5 service nova] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:26:16.477410 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 696/1382] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:26:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:26:18.537063 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327df762-fdcb-47d5-a30e-73ebf73e8d5e None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:26:18.537758 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327df762-fdcb-47d5-a30e-73ebf73e8d5e None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:26:18.537758 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327df762-fdcb-47d5-a30e-73ebf73e8d5e None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:26:18.539051 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 687/1383] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:26:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:26:18.544944 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f988fd8-2719-496d-a009-9ac4ee10dabf None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:26:18.545509 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f988fd8-2719-496d-a009-9ac4ee10dabf None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:26:18.545625 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f988fd8-2719-496d-a009-9ac4ee10dabf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:26:18.628773 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-4f988fd8-2719-496d-a009-9ac4ee10dabf None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:26:18.630472 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-4f988fd8-2719-496d-a009-9ac4ee10dabf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:26:18.633116 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:26:18.735478 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 697/1384] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:26:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:26:18.778773 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327df762-fdcb-47d5-a30e-73ebf73e8d5e None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:26:18.780722 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-327df762-fdcb-47d5-a30e-73ebf73e8d5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:26:18.956065 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327df762-fdcb-47d5-a30e-73ebf73e8d5e None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:26:19.036523 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:26:19.089063 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5306c4fd-b087-4f90-8288-56a50fc45b53 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87adc1d5bc240e99f95f6595b350be4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:26:19.089697 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5306c4fd-b087-4f90-8288-56a50fc45b53 service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:26:19.089981 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5306c4fd-b087-4f90-8288-56a50fc45b53 service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:26:19.090256 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5306c4fd-b087-4f90-8288-56a50fc45b53 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:26:19.091903 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-5306c4fd-b087-4f90-8288-56a50fc45b53 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:26:19.185687 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5306c4fd-b087-4f90-8288-56a50fc45b53 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:26:19.187371 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5306c4fd-b087-4f90-8288-56a50fc45b53 service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:26:19.207523 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 688/1385] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:26:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:26:21.218014 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a63981-f478-42fc-aad6-6f9c9a2f71fc None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:26:21.218548 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a63981-f478-42fc-aad6-6f9c9a2f71fc None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:26:21.218679 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a63981-f478-42fc-aad6-6f9c9a2f71fc None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:26:21.219966 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 698/1386] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:26:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 01:26:21.225665 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14188c2-139d-4113-9992-a08da0b4711d None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:26:21.226266 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14188c2-139d-4113-9992-a08da0b4711d None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:26:21.226266 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14188c2-139d-4113-9992-a08da0b4711d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:26:21.306725 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-b14188c2-139d-4113-9992-a08da0b4711d None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:26:21.308278 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-b14188c2-139d-4113-9992-a08da0b4711d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:26:21.341098 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:26:21.428315 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 689/1387] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:26:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:26:21.474582 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a63981-f478-42fc-aad6-6f9c9a2f71fc None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:26:21.476783 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-62a63981-f478-42fc-aad6-6f9c9a2f71fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:26:21.625010 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:26:21.646751 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a63981-f478-42fc-aad6-6f9c9a2f71fc None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:26:21.787556 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e5cf8d-9310-49f0-afaf-893fa0fbc299 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87adc1d5bc240e99f95f6595b350be4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:26:21.788071 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e5cf8d-9310-49f0-afaf-893fa0fbc299 service nova] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:26:21.788347 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e5cf8d-9310-49f0-afaf-893fa0fbc299 service nova] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:26:21.788620 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e5cf8d-9310-49f0-afaf-893fa0fbc299 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:26:21.790299 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-c5e5cf8d-9310-49f0-afaf-893fa0fbc299 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:26:21.887057 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e5cf8d-9310-49f0-afaf-893fa0fbc299 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:26:21.888662 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e5cf8d-9310-49f0-afaf-893fa0fbc299 service nova] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:26:21.893892 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:26:21.909197 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 699/1388] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:26:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:26:24.151202 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80080c9-cd25-4a57-a845-1c41bec3ef76 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:26:24.151775 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80080c9-cd25-4a57-a845-1c41bec3ef76 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:26:24.151775 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80080c9-cd25-4a57-a845-1c41bec3ef76 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:26:24.153194 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 690/1389] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:26:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:26:24.158624 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8561548e-5824-4d55-985f-7bd4982e5f0b None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:26:24.159030 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8561548e-5824-4d55-985f-7bd4982e5f0b None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:26:24.159187 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8561548e-5824-4d55-985f-7bd4982e5f0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:26:24.226237 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-8561548e-5824-4d55-985f-7bd4982e5f0b None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:26:24.227479 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-8561548e-5824-4d55-985f-7bd4982e5f0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:26:24.326204 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 700/1390] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:26:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:26:24.371219 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80080c9-cd25-4a57-a845-1c41bec3ef76 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:26:24.372847 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-a80080c9-cd25-4a57-a845-1c41bec3ef76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:26:24.519411 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80080c9-cd25-4a57-a845-1c41bec3ef76 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:26:24.529814 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:26:24.636514 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de2e58e-12f1-4190-bf10-061b49e0937c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87adc1d5bc240e99f95f6595b350be4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:26:24.637165 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de2e58e-12f1-4190-bf10-061b49e0937c service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:26:24.637450 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de2e58e-12f1-4190-bf10-061b49e0937c service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:26:24.637723 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de2e58e-12f1-4190-bf10-061b49e0937c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:26:24.639336 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-1de2e58e-12f1-4190-bf10-061b49e0937c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:26:24.728737 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de2e58e-12f1-4190-bf10-061b49e0937c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:26:24.730671 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de2e58e-12f1-4190-bf10-061b49e0937c service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:26:24.751939 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 691/1391] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:26:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:26:25.697779 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2312eb8-a289-4b04-a674-d6528400137f None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:26:25.698355 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2312eb8-a289-4b04-a674-d6528400137f None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:26:25.698355 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2312eb8-a289-4b04-a674-d6528400137f None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:26:25.699676 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 701/1392] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:26:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:26:25.704953 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d2d23c-217c-4bb2-9c93-f86800f77340 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:26:25.705417 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d2d23c-217c-4bb2-9c93-f86800f77340 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:26:25.705752 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d2d23c-217c-4bb2-9c93-f86800f77340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:26:25.722207 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:26:25.774881 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-03d2d23c-217c-4bb2-9c93-f86800f77340 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:26:25.776097 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-03d2d23c-217c-4bb2-9c93-f86800f77340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:26:25.872413 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 692/1393] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:26:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:26:25.929771 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f34243c-298b-4930-87b2-ab9be71c351e None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:26:25.930051 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f34243c-298b-4930-87b2-ab9be71c351e None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:26:25.930326 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f34243c-298b-4930-87b2-ab9be71c351e None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:26:25.931787 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 702/1394] 10.222.0.26 () {54 vars in 966 bytes} [Sat Aug 6 01:26:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:26:25.936601 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f28afef-89fc-4853-8044-6ad10ebd852f None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:26:25.936891 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f28afef-89fc-4853-8044-6ad10ebd852f None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:26:25.937201 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f28afef-89fc-4853-8044-6ad10ebd852f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:26:26.013139 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-7f28afef-89fc-4853-8044-6ad10ebd852f None None] MFA Rules not processed for user `abb76a5c8ef74bf2b18c56074daaad58`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:26:26.014256 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-7f28afef-89fc-4853-8044-6ad10ebd852f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:26:26.047830 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:26:26.091067 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 693/1395] 10.222.0.26 () {58 vars in 1063 bytes} [Sat Aug 6 01:26:25 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:26:26.099384 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f34243c-298b-4930-87b2-ab9be71c351e None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:26:26.101422 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-7f34243c-298b-4930-87b2-ab9be71c351e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:26:26.172765 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:26:26.204295 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f34243c-298b-4930-87b2-ab9be71c351e None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:26:26.277910 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de58b7e2-aac0-4e77-846e-41224fe80df1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abb76a5c8ef74bf2b18c56074daaad58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:26:26.278454 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de58b7e2-aac0-4e77-846e-41224fe80df1 service glance] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:26:26.278735 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de58b7e2-aac0-4e77-846e-41224fe80df1 service glance] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:26:26.279011 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de58b7e2-aac0-4e77-846e-41224fe80df1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:26:26.280631 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-de58b7e2-aac0-4e77-846e-41224fe80df1 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:26:26.391196 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de58b7e2-aac0-4e77-846e-41224fe80df1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:26:26.392700 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de58b7e2-aac0-4e77-846e-41224fe80df1 service glance] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:26:26.412344 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:26:26.416162 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 703/1396] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:26:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:26:26.806643 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d4e4ef-2332-4f12-99b3-c5bbdd0c0b12 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:26:26.807168 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d4e4ef-2332-4f12-99b3-c5bbdd0c0b12 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:26:26.807330 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d4e4ef-2332-4f12-99b3-c5bbdd0c0b12 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:26:26.808591 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 694/1397] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:26:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:26:26.814227 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d40ead4-0ff5-4758-932b-6aaf44b05c2c None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:26:26.815113 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d40ead4-0ff5-4758-932b-6aaf44b05c2c None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:26:26.815573 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d40ead4-0ff5-4758-932b-6aaf44b05c2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:26:26.883810 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-9d40ead4-0ff5-4758-932b-6aaf44b05c2c None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:26:26.885061 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-9d40ead4-0ff5-4758-932b-6aaf44b05c2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:26:26.985470 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 704/1398] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:26:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:26:27.033173 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d4e4ef-2332-4f12-99b3-c5bbdd0c0b12 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:26:27.034771 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-c4d4e4ef-2332-4f12-99b3-c5bbdd0c0b12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:26:27.161257 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76584927-2009-46f9-95b0-d1a0ad048371 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:26:27.161551 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76584927-2009-46f9-95b0-d1a0ad048371 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:26:27.161834 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76584927-2009-46f9-95b0-d1a0ad048371 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:26:27.197138 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d4e4ef-2332-4f12-99b3-c5bbdd0c0b12 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:26:27.231005 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-76584927-2009-46f9-95b0-d1a0ad048371 None None] MFA Rules not processed for user `8e86f38590824894a3b960e6774ff8a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:26:27.232618 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-76584927-2009-46f9-95b0-d1a0ad048371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:26:27.237329 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:26:27.263368 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:26:27.311659 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85478f4b-b479-401d-aaab-af8883f35d9b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87adc1d5bc240e99f95f6595b350be4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:26:27.312192 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85478f4b-b479-401d-aaab-af8883f35d9b service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:26:27.312473 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85478f4b-b479-401d-aaab-af8883f35d9b service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:26:27.312747 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85478f4b-b479-401d-aaab-af8883f35d9b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:26:27.314400 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-85478f4b-b479-401d-aaab-af8883f35d9b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:26:27.322561 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 705/1399] 10.222.0.26 () {58 vars in 1009 bytes} [Sat Aug 6 01:26:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:26:27.371947 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddfe6a0-e53b-4ba6-a2f0-00385d6c4772 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:26:27.372237 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddfe6a0-e53b-4ba6-a2f0-00385d6c4772 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:26:27.372517 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddfe6a0-e53b-4ba6-a2f0-00385d6c4772 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:26:27.373900 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 706/1400] 10.222.0.26 () {54 vars in 943 bytes} [Sat Aug 6 01:26:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 01:26:27.380130 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57b0de0-901f-4e63-a830-28848e0d4c2d None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:26:27.380489 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57b0de0-901f-4e63-a830-28848e0d4c2d None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:26:27.380834 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57b0de0-901f-4e63-a830-28848e0d4c2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:26:27.410420 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85478f4b-b479-401d-aaab-af8883f35d9b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:26:27.412242 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85478f4b-b479-401d-aaab-af8883f35d9b service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:26:27.431429 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 695/1401] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:26:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:26:27.466103 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-b57b0de0-901f-4e63-a830-28848e0d4c2d None None] MFA Rules not processed for user `2b23f4478f49471fba6f43e6e4be99da`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:26:27.467301 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-b57b0de0-901f-4e63-a830-28848e0d4c2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:26:27.611655 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 707/1402] 10.222.0.26 () {58 vars in 1040 bytes} [Sat Aug 6 01:26:27 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:26:27.622526 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85478f4b-b479-401d-aaab-af8883f35d9b service nova] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:26:27.623778 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-85478f4b-b479-401d-aaab-af8883f35d9b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:26:27.680796 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:26:27.729228 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85478f4b-b479-401d-aaab-af8883f35d9b service nova] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:26:27.966697 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7e00c0-a534-48e5-b9fa-86d93975a3e4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b23f4478f49471fba6f43e6e4be99da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:26:27.967384 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7e00c0-a534-48e5-b9fa-86d93975a3e4 service swift] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:26:27.967569 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7e00c0-a534-48e5-b9fa-86d93975a3e4 service swift] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:26:27.967837 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7e00c0-a534-48e5-b9fa-86d93975a3e4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:26:27.969494 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-9a7e00c0-a534-48e5-b9fa-86d93975a3e4 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:26:28.050141 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:26:28.054664 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7e00c0-a534-48e5-b9fa-86d93975a3e4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:26:28.056496 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7e00c0-a534-48e5-b9fa-86d93975a3e4 service swift] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:26:28.064522 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 696/1403] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Aug 6 01:26:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 06 01:26:29.516707 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57894a66-c092-4206-935a-33772230c29b None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:26:29.517256 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57894a66-c092-4206-935a-33772230c29b None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:26:29.517366 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57894a66-c092-4206-935a-33772230c29b None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:26:29.518665 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 708/1404] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:26:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:26:29.524455 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f11b270-32a8-460d-89b0-9d67483e41a7 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:26:29.524878 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f11b270-32a8-460d-89b0-9d67483e41a7 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:26:29.525010 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f11b270-32a8-460d-89b0-9d67483e41a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:26:29.605757 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-1f11b270-32a8-460d-89b0-9d67483e41a7 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:26:29.606973 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-1f11b270-32a8-460d-89b0-9d67483e41a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:26:29.711787 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 697/1405] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:26:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:26:29.762568 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57894a66-c092-4206-935a-33772230c29b None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:26:29.764445 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-57894a66-c092-4206-935a-33772230c29b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:26:29.769514 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:26:29.952647 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57894a66-c092-4206-935a-33772230c29b None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:26:30.060060 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:26:30.136244 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0bead2-a129-42e9-a665-d1368d7dc483 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87adc1d5bc240e99f95f6595b350be4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:26:30.136925 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0bead2-a129-42e9-a665-d1368d7dc483 service nova] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:26:30.137266 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0bead2-a129-42e9-a665-d1368d7dc483 service nova] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:26:30.137550 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0bead2-a129-42e9-a665-d1368d7dc483 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:26:30.141680 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-4f0bead2-a129-42e9-a665-d1368d7dc483 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:26:30.247764 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0bead2-a129-42e9-a665-d1368d7dc483 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:26:30.249475 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0bead2-a129-42e9-a665-d1368d7dc483 service nova] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:26:30.270001 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 709/1406] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:26:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:26:32.325686 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d763eb-99a7-455d-8a68-4f819952e209 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:26:32.326217 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d763eb-99a7-455d-8a68-4f819952e209 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:26:32.326217 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d763eb-99a7-455d-8a68-4f819952e209 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:26:32.327593 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 698/1407] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:26:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:26:32.333536 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eafa5bf-4665-49f7-8cb9-4bd98b88faa0 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:26:32.333949 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eafa5bf-4665-49f7-8cb9-4bd98b88faa0 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:26:32.334215 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eafa5bf-4665-49f7-8cb9-4bd98b88faa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:26:32.413014 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-9eafa5bf-4665-49f7-8cb9-4bd98b88faa0 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:26:32.414939 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-9eafa5bf-4665-49f7-8cb9-4bd98b88faa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:26:32.459045 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:26:32.527912 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 710/1408] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:26:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:26:32.575890 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d763eb-99a7-455d-8a68-4f819952e209 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:26:32.577456 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-90d763eb-99a7-455d-8a68-4f819952e209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:26:32.660040 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:26:32.741309 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d763eb-99a7-455d-8a68-4f819952e209 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:26:32.870567 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de71fe63-0940-4c52-b6b6-16d56692a185 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87adc1d5bc240e99f95f6595b350be4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:26:32.871102 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de71fe63-0940-4c52-b6b6-16d56692a185 service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:26:32.871385 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de71fe63-0940-4c52-b6b6-16d56692a185 service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:26:32.871661 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de71fe63-0940-4c52-b6b6-16d56692a185 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:26:32.873329 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-de71fe63-0940-4c52-b6b6-16d56692a185 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:26:32.911560 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:26:32.971655 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de71fe63-0940-4c52-b6b6-16d56692a185 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:26:32.973386 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de71fe63-0940-4c52-b6b6-16d56692a185 service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:26:32.994761 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 699/1409] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:26:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:26:42.526041 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=26 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:26:42.992370 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=30 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:26:50.719734 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2f9260-445e-48b9-884f-96958c569f13 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:26:50.720322 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2f9260-445e-48b9-884f-96958c569f13 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:26:50.720322 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2f9260-445e-48b9-884f-96958c569f13 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:26:50.721678 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 711/1410] 10.222.0.26 () {54 vars in 918 bytes} [Sat Aug 6 01:26:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:26:50.727455 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be2c2dc-e0fa-441c-bb34-21862556a1aa None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:26:50.727886 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be2c2dc-e0fa-441c-bb34-21862556a1aa None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:26:50.728015 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be2c2dc-e0fa-441c-bb34-21862556a1aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:26:50.803913 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-2be2c2dc-e0fa-441c-bb34-21862556a1aa None None] MFA Rules not processed for user `fa3b93ddd9464a2cba724d67b56b477a`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:26:50.805139 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-2be2c2dc-e0fa-441c-bb34-21862556a1aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:26:50.907801 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 700/1411] 10.222.0.26 () {58 vars in 1015 bytes} [Sat Aug 6 01:26:50 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:26:50.922781 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aac87a-95d0-4d42-98df-0ce828ba5d61 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:26:50.923068 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aac87a-95d0-4d42-98df-0ce828ba5d61 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:26:50.923345 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aac87a-95d0-4d42-98df-0ce828ba5d61 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:26:50.924669 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 712/1412] 10.222.0.26 () {54 vars in 918 bytes} [Sat Aug 6 01:26:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:26:50.929883 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4323da-b440-47a9-b19b-fab06322f0e7 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:26:50.930173 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4323da-b440-47a9-b19b-fab06322f0e7 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:26:50.930451 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4323da-b440-47a9-b19b-fab06322f0e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:26:51.003161 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:26:51.006431 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-0a4323da-b440-47a9-b19b-fab06322f0e7 None None] MFA Rules not processed for user `b87adc1d5bc240e99f95f6595b350be4`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:26:51.008157 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-0a4323da-b440-47a9-b19b-fab06322f0e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:26:51.123273 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 701/1413] 10.222.0.26 () {58 vars in 1015 bytes} [Sat Aug 6 01:26:50 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:26:53.589797 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e3c8a7-c63d-447d-b0e3-0d97b06eeda3 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:26:53.590356 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e3c8a7-c63d-447d-b0e3-0d97b06eeda3 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:26:53.590356 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e3c8a7-c63d-447d-b0e3-0d97b06eeda3 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:26:53.591690 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 713/1414] 127.0.0.1 () {54 vars in 917 bytes} [Sat Aug 6 01:26:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:26:53.597037 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eee3c4f-f2f8-4e2e-a851-7b14c9e707d3 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:26:53.597454 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eee3c4f-f2f8-4e2e-a851-7b14c9e707d3 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:26:53.597611 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eee3c4f-f2f8-4e2e-a851-7b14c9e707d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:26:53.662148 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-2eee3c4f-f2f8-4e2e-a851-7b14c9e707d3 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:26:53.663258 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-2eee3c4f-f2f8-4e2e-a851-7b14c9e707d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:26:53.716823 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:26:53.758032 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 702/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Aug 6 01:26:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:26:53.801364 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e3c8a7-c63d-447d-b0e3-0d97b06eeda3 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:26:53.802858 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-23e3c8a7-c63d-447d-b0e3-0d97b06eeda3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:26:53.945846 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e3c8a7-c63d-447d-b0e3-0d97b06eeda3 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:26:54.016896 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:26:54.057669 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416c80af-fce8-4655-8f3e-4968073e5170 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87adc1d5bc240e99f95f6595b350be4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:26:54.058293 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416c80af-fce8-4655-8f3e-4968073e5170 service nova] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:26:54.058578 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416c80af-fce8-4655-8f3e-4968073e5170 service nova] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:26:54.058882 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416c80af-fce8-4655-8f3e-4968073e5170 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:26:54.060512 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-416c80af-fce8-4655-8f3e-4968073e5170 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:26:54.149861 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416c80af-fce8-4655-8f3e-4968073e5170 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:26:54.151446 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416c80af-fce8-4655-8f3e-4968073e5170 service nova] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:26:54.170697 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 714/1416] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:26:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:26:55.065823 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d06c71-9e40-495f-9f1e-497db1945db3 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:26:55.066368 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d06c71-9e40-495f-9f1e-497db1945db3 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:26:55.066368 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d06c71-9e40-495f-9f1e-497db1945db3 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:26:55.068425 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 703/1417] 127.0.0.1 () {54 vars in 901 bytes} [Sat Aug 6 01:26:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:26:55.073486 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d66362-2d6d-4e03-91e9-3755938518e1 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:26:55.073914 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d66362-2d6d-4e03-91e9-3755938518e1 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:26:55.074050 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d66362-2d6d-4e03-91e9-3755938518e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:26:55.152251 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-52d66362-2d6d-4e03-91e9-3755938518e1 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:26:55.153508 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-52d66362-2d6d-4e03-91e9-3755938518e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:26:55.182844 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:26:55.277448 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 715/1418] 127.0.0.1 () {58 vars in 998 bytes} [Sat Aug 6 01:26:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:26:55.437551 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d06c71-9e40-495f-9f1e-497db1945db3 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:26:55.439087 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-03d06c71-9e40-495f-9f1e-497db1945db3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:26:55.609847 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d06c71-9e40-495f-9f1e-497db1945db3 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:26:55.654721 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:26:55.737559 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d49259-72b2-4f29-be63-f874b29912dc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87adc1d5bc240e99f95f6595b350be4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:26:55.738182 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d49259-72b2-4f29-be63-f874b29912dc service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:26:55.738467 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d49259-72b2-4f29-be63-f874b29912dc service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:26:55.738748 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d49259-72b2-4f29-be63-f874b29912dc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:26:55.740518 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-51d49259-72b2-4f29-be63-f874b29912dc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:26:55.837886 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d49259-72b2-4f29-be63-f874b29912dc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:26:55.840600 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d49259-72b2-4f29-be63-f874b29912dc service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:26:55.865487 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 704/1419] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:26:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:26:56.756340 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9735463c-54ec-4983-a40b-48a0f811851c None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:26:56.756937 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9735463c-54ec-4983-a40b-48a0f811851c None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:26:56.756937 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9735463c-54ec-4983-a40b-48a0f811851c None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:26:56.758311 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 716/1420] 127.0.0.1 () {54 vars in 901 bytes} [Sat Aug 6 01:26:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:26:56.763322 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f3909f-5d24-4070-b148-c816b222ff11 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:26:56.763780 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f3909f-5d24-4070-b148-c816b222ff11 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:26:56.763897 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f3909f-5d24-4070-b148-c816b222ff11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:26:56.818932 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:26:56.840544 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-02f3909f-5d24-4070-b148-c816b222ff11 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:26:56.841803 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-02f3909f-5d24-4070-b148-c816b222ff11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:26:56.936801 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 705/1421] 127.0.0.1 () {58 vars in 998 bytes} [Sat Aug 6 01:26:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:26:57.090686 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9735463c-54ec-4983-a40b-48a0f811851c None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:26:57.092550 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-9735463c-54ec-4983-a40b-48a0f811851c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:26:57.238675 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9735463c-54ec-4983-a40b-48a0f811851c None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:26:57.250642 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:26:57.350709 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3b221b-389b-45bb-a7a2-13509ebb5881 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87adc1d5bc240e99f95f6595b350be4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:26:57.351245 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3b221b-389b-45bb-a7a2-13509ebb5881 service nova] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:26:57.351522 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3b221b-389b-45bb-a7a2-13509ebb5881 service nova] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:26:57.351790 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3b221b-389b-45bb-a7a2-13509ebb5881 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:26:57.353445 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-fc3b221b-389b-45bb-a7a2-13509ebb5881 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:26:57.441321 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3b221b-389b-45bb-a7a2-13509ebb5881 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:26:57.442926 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3b221b-389b-45bb-a7a2-13509ebb5881 service nova] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:26:57.461162 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 717/1422] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:26:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:26:58.322931 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fc74e7-9126-4249-92cf-5766dd099523 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:26:58.323517 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fc74e7-9126-4249-92cf-5766dd099523 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:26:58.323517 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fc74e7-9126-4249-92cf-5766dd099523 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:26:58.324851 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 706/1423] 127.0.0.1 () {54 vars in 901 bytes} [Sat Aug 6 01:26:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:26:58.329807 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf76559b-eb27-4bfc-993d-397905d9c71e None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:26:58.330206 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf76559b-eb27-4bfc-993d-397905d9c71e None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:26:58.330363 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf76559b-eb27-4bfc-993d-397905d9c71e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:26:58.350299 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:26:58.408543 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-bf76559b-eb27-4bfc-993d-397905d9c71e None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:26:58.409792 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-bf76559b-eb27-4bfc-993d-397905d9c71e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:26:58.505784 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 718/1424] 127.0.0.1 () {58 vars in 998 bytes} [Sat Aug 6 01:26:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:26:58.654611 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fc74e7-9126-4249-92cf-5766dd099523 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:26:58.656088 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-b8fc74e7-9126-4249-92cf-5766dd099523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:26:58.765841 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:26:58.801959 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fc74e7-9126-4249-92cf-5766dd099523 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:26:58.913383 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c4a32a-f645-43c6-a681-52fe56e215fc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87adc1d5bc240e99f95f6595b350be4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:26:58.913922 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c4a32a-f645-43c6-a681-52fe56e215fc service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:26:58.914203 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c4a32a-f645-43c6-a681-52fe56e215fc service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:26:58.914479 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c4a32a-f645-43c6-a681-52fe56e215fc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:26:58.916104 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-00c4a32a-f645-43c6-a681-52fe56e215fc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:26:58.991243 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:26:59.005399 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c4a32a-f645-43c6-a681-52fe56e215fc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:26:59.007176 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c4a32a-f645-43c6-a681-52fe56e215fc service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:26:59.025015 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 707/1425] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:26:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:26:59.890691 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19e2fee-9a77-496c-923d-df68663ed596 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:26:59.891255 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19e2fee-9a77-496c-923d-df68663ed596 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:26:59.891255 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19e2fee-9a77-496c-923d-df68663ed596 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:26:59.892601 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 719/1426] 127.0.0.1 () {54 vars in 901 bytes} [Sat Aug 6 01:26:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:26:59.898169 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eb52f0-b1b9-4584-b35c-dea78012188d None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:26:59.898567 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eb52f0-b1b9-4584-b35c-dea78012188d None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:26:59.898731 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eb52f0-b1b9-4584-b35c-dea78012188d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:26:59.975601 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-78eb52f0-b1b9-4584-b35c-dea78012188d None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:26:59.976810 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-78eb52f0-b1b9-4584-b35c-dea78012188d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:27:00.074721 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 708/1427] 127.0.0.1 () {58 vars in 998 bytes} [Sat Aug 6 01:26:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:27:00.229225 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19e2fee-9a77-496c-923d-df68663ed596 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:27:00.231038 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-e19e2fee-9a77-496c-923d-df68663ed596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:27:00.315147 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:27:00.377254 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19e2fee-9a77-496c-923d-df68663ed596 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:27:00.488809 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf9cde7-a706-4e79-8faa-b0d25341fe74 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87adc1d5bc240e99f95f6595b350be4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:27:00.489448 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf9cde7-a706-4e79-8faa-b0d25341fe74 service nova] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:27:00.489760 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf9cde7-a706-4e79-8faa-b0d25341fe74 service nova] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:27:00.490041 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf9cde7-a706-4e79-8faa-b0d25341fe74 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:27:00.491673 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-ddf9cde7-a706-4e79-8faa-b0d25341fe74 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:27:00.539400 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:27:00.580827 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf9cde7-a706-4e79-8faa-b0d25341fe74 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:27:00.582476 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf9cde7-a706-4e79-8faa-b0d25341fe74 service nova] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:27:00.601061 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 720/1428] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:27:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:27:01.402257 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88fd545-0265-4fb8-9339-471e4ed76fdd None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:27:01.402722 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88fd545-0265-4fb8-9339-471e4ed76fdd None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:27:01.402851 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88fd545-0265-4fb8-9339-471e4ed76fdd None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:27:01.404184 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 709/1429] 127.0.0.1 () {54 vars in 901 bytes} [Sat Aug 6 01:27:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:27:01.409391 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2df4a5-3fe6-42a8-9728-842a01d3bddf None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:27:01.409838 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2df4a5-3fe6-42a8-9728-842a01d3bddf None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:27:01.409950 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2df4a5-3fe6-42a8-9728-842a01d3bddf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:27:01.488291 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-9a2df4a5-3fe6-42a8-9728-842a01d3bddf None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:27:01.489542 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-9a2df4a5-3fe6-42a8-9728-842a01d3bddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:27:01.585709 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 721/1430] 127.0.0.1 () {58 vars in 998 bytes} [Sat Aug 6 01:27:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:27:01.741762 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88fd545-0265-4fb8-9339-471e4ed76fdd None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:27:01.743288 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-a88fd545-0265-4fb8-9339-471e4ed76fdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:27:01.791756 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:27:01.891035 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88fd545-0265-4fb8-9339-471e4ed76fdd None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:27:02.005557 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1102f994-5212-49a9-bc14-472423b483da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87adc1d5bc240e99f95f6595b350be4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:27:02.006104 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1102f994-5212-49a9-bc14-472423b483da service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:27:02.006388 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1102f994-5212-49a9-bc14-472423b483da service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:27:02.006751 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1102f994-5212-49a9-bc14-472423b483da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:27:02.008421 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-1102f994-5212-49a9-bc14-472423b483da service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:27:02.015319 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:27:02.100055 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1102f994-5212-49a9-bc14-472423b483da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:27:02.101691 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1102f994-5212-49a9-bc14-472423b483da service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:27:02.120261 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 710/1431] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:27:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:27:02.922818 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa97bb6-3646-4561-b963-bc78da370be7 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:27:02.923270 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa97bb6-3646-4561-b963-bc78da370be7 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:27:02.923380 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa97bb6-3646-4561-b963-bc78da370be7 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:27:02.924737 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 722/1432] 127.0.0.1 () {54 vars in 901 bytes} [Sat Aug 6 01:27:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:27:02.929725 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfb672e-69f9-43cb-afd4-870432a7585f None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:27:02.930164 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfb672e-69f9-43cb-afd4-870432a7585f None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:27:02.930276 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfb672e-69f9-43cb-afd4-870432a7585f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:27:03.006644 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-ddfb672e-69f9-43cb-afd4-870432a7585f None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:27:03.007866 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-ddfb672e-69f9-43cb-afd4-870432a7585f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:27:03.073858 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:27:03.103949 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 711/1433] 127.0.0.1 () {58 vars in 998 bytes} [Sat Aug 6 01:27:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:27:03.255218 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa97bb6-3646-4561-b963-bc78da370be7 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:27:03.256760 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-faa97bb6-3646-4561-b963-bc78da370be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:27:03.274519 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:27:03.402930 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa97bb6-3646-4561-b963-bc78da370be7 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:27:03.490646 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:27:03.516994 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1798d5c7-578c-466e-9e6e-8e3952f1b3c0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87adc1d5bc240e99f95f6595b350be4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:27:03.517574 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1798d5c7-578c-466e-9e6e-8e3952f1b3c0 service nova] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:27:03.517857 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1798d5c7-578c-466e-9e6e-8e3952f1b3c0 service nova] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:27:03.518132 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1798d5c7-578c-466e-9e6e-8e3952f1b3c0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:27:03.519743 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-1798d5c7-578c-466e-9e6e-8e3952f1b3c0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:27:03.608069 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1798d5c7-578c-466e-9e6e-8e3952f1b3c0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:27:03.609710 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1798d5c7-578c-466e-9e6e-8e3952f1b3c0 service nova] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:27:03.627952 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 723/1434] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:27:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:27:04.430201 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83245de7-208b-42ca-97ca-c28090ebfd02 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:27:04.430697 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83245de7-208b-42ca-97ca-c28090ebfd02 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:27:04.430808 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83245de7-208b-42ca-97ca-c28090ebfd02 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:27:04.432208 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 712/1435] 127.0.0.1 () {54 vars in 901 bytes} [Sat Aug 6 01:27:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:27:04.437104 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfba6dd-d164-474d-86ed-bb0255879472 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:27:04.437569 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfba6dd-d164-474d-86ed-bb0255879472 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:27:04.437677 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfba6dd-d164-474d-86ed-bb0255879472 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:27:04.512844 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-6bfba6dd-d164-474d-86ed-bb0255879472 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:27:04.514087 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-6bfba6dd-d164-474d-86ed-bb0255879472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:27:04.555094 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:27:04.609064 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 724/1436] 127.0.0.1 () {58 vars in 998 bytes} [Sat Aug 6 01:27:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:27:04.763711 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83245de7-208b-42ca-97ca-c28090ebfd02 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:27:04.765280 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-83245de7-208b-42ca-97ca-c28090ebfd02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:27:04.910080 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83245de7-208b-42ca-97ca-c28090ebfd02 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:27:04.963670 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:27:05.022193 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159e9408-58af-471f-8685-eb98899a5841 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87adc1d5bc240e99f95f6595b350be4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:27:05.022804 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159e9408-58af-471f-8685-eb98899a5841 service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:27:05.023083 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159e9408-58af-471f-8685-eb98899a5841 service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:27:05.023356 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159e9408-58af-471f-8685-eb98899a5841 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:27:05.024964 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-159e9408-58af-471f-8685-eb98899a5841 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:27:05.114925 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159e9408-58af-471f-8685-eb98899a5841 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:27:05.116518 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159e9408-58af-471f-8685-eb98899a5841 service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:27:05.134883 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 713/1437] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:27:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:27:05.947785 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aa5c5b-b70f-4f82-b967-04adf21dbce1 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:27:05.948350 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aa5c5b-b70f-4f82-b967-04adf21dbce1 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:27:05.948350 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aa5c5b-b70f-4f82-b967-04adf21dbce1 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:27:05.949755 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 725/1438] 127.0.0.1 () {54 vars in 901 bytes} [Sat Aug 6 01:27:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:27:05.954715 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dffde5f-2266-4041-9010-17dd86d533ef None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:27:05.955098 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dffde5f-2266-4041-9010-17dd86d533ef None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:27:05.955275 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dffde5f-2266-4041-9010-17dd86d533ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:27:06.034456 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-3dffde5f-2266-4041-9010-17dd86d533ef None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:27:06.035699 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-3dffde5f-2266-4041-9010-17dd86d533ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:27:06.038277 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:27:06.135024 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 714/1439] 127.0.0.1 () {58 vars in 998 bytes} [Sat Aug 6 01:27:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:27:06.285173 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3aa5c5b-b70f-4f82-b967-04adf21dbce1 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:27:06.286841 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-e3aa5c5b-b70f-4f82-b967-04adf21dbce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:27:06.432843 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3aa5c5b-b70f-4f82-b967-04adf21dbce1 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:27:06.460026 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:27:06.544725 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8c1444-ffb7-41df-8a78-8297e5a34369 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87adc1d5bc240e99f95f6595b350be4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:27:06.545290 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8c1444-ffb7-41df-8a78-8297e5a34369 service nova] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:27:06.545566 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8c1444-ffb7-41df-8a78-8297e5a34369 service nova] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:27:06.545837 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8c1444-ffb7-41df-8a78-8297e5a34369 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:27:06.547431 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-1b8c1444-ffb7-41df-8a78-8297e5a34369 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:27:06.636146 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8c1444-ffb7-41df-8a78-8297e5a34369 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:27:06.637828 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8c1444-ffb7-41df-8a78-8297e5a34369 service nova] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:27:06.655481 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 726/1440] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:27:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:27:08.568402 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511bd375-6ae4-4d8b-ac07-80cac1d22f8b None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:27:08.568928 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511bd375-6ae4-4d8b-ac07-80cac1d22f8b None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:27:08.568928 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511bd375-6ae4-4d8b-ac07-80cac1d22f8b None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:27:08.570344 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 715/1441] 127.0.0.1 () {54 vars in 917 bytes} [Sat Aug 6 01:27:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:27:08.575628 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baf17ea-cd1b-4b93-af4b-4e933bfc6660 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:27:08.576023 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baf17ea-cd1b-4b93-af4b-4e933bfc6660 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:27:08.576183 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baf17ea-cd1b-4b93-af4b-4e933bfc6660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:27:08.616781 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:27:08.642457 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-8baf17ea-cd1b-4b93-af4b-4e933bfc6660 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:27:08.643666 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-8baf17ea-cd1b-4b93-af4b-4e933bfc6660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:27:08.740087 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 727/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Aug 6 01:27:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:27:08.779450 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511bd375-6ae4-4d8b-ac07-80cac1d22f8b None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:27:08.780975 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-511bd375-6ae4-4d8b-ac07-80cac1d22f8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:27:08.883536 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511bd375-6ae4-4d8b-ac07-80cac1d22f8b None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:27:08.915512 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:27:08.956590 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c1a3f1-56ab-43ff-81c9-3fced472b3b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:27:08.957238 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c1a3f1-56ab-43ff-81c9-3fced472b3b4 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:27:08.957525 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c1a3f1-56ab-43ff-81c9-3fced472b3b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:27:08.958450 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c1a3f1-56ab-43ff-81c9-3fced472b3b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:27:08.960098 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-29c1a3f1-56ab-43ff-81c9-3fced472b3b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:27:09.072544 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c1a3f1-56ab-43ff-81c9-3fced472b3b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:27:09.074015 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c1a3f1-56ab-43ff-81c9-3fced472b3b4 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:27:09.095537 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 716/1443] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:27:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:27:11.227619 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5afcb3d-faba-4065-8ae5-f27d0609e496 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:27:11.228169 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5afcb3d-faba-4065-8ae5-f27d0609e496 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:27:11.228169 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5afcb3d-faba-4065-8ae5-f27d0609e496 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:27:11.229546 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 728/1444] 127.0.0.1 () {54 vars in 917 bytes} [Sat Aug 6 01:27:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:27:11.234922 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524f35a6-1200-4d81-bca2-fb0209b10591 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:27:11.235341 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524f35a6-1200-4d81-bca2-fb0209b10591 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:27:11.235468 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524f35a6-1200-4d81-bca2-fb0209b10591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:27:11.308129 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-524f35a6-1200-4d81-bca2-fb0209b10591 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:27:11.309353 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-524f35a6-1200-4d81-bca2-fb0209b10591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:27:11.321743 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:27:11.405038 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 717/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Aug 6 01:27:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:27:11.444469 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5afcb3d-faba-4065-8ae5-f27d0609e496 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:27:11.446052 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-a5afcb3d-faba-4065-8ae5-f27d0609e496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:27:11.548228 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5afcb3d-faba-4065-8ae5-f27d0609e496 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:27:11.560058 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:27:11.620575 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78eaa87-5cb7-4319-801c-27c76a487469 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:27:11.621223 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78eaa87-5cb7-4319-801c-27c76a487469 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:27:11.621506 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78eaa87-5cb7-4319-801c-27c76a487469 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:27:11.621784 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78eaa87-5cb7-4319-801c-27c76a487469 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:27:11.623400 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-e78eaa87-5cb7-4319-801c-27c76a487469 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:27:11.729200 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78eaa87-5cb7-4319-801c-27c76a487469 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:27:11.730638 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78eaa87-5cb7-4319-801c-27c76a487469 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:27:11.748913 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 729/1446] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:27:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:27:14.130291 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef451db-91d6-4cad-a6e3-5dbbfba1ccee None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:27:14.130843 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef451db-91d6-4cad-a6e3-5dbbfba1ccee None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:27:14.130843 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef451db-91d6-4cad-a6e3-5dbbfba1ccee None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:27:14.132214 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 718/1447] 127.0.0.1 () {54 vars in 917 bytes} [Sat Aug 6 01:27:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:27:14.137339 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c100241c-5e3d-4c69-86ac-d839a575e8b4 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:27:14.137764 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c100241c-5e3d-4c69-86ac-d839a575e8b4 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:27:14.137904 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c100241c-5e3d-4c69-86ac-d839a575e8b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:27:14.185685 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:27:14.204207 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-c100241c-5e3d-4c69-86ac-d839a575e8b4 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:27:14.205451 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-c100241c-5e3d-4c69-86ac-d839a575e8b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:27:14.300971 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 730/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Aug 6 01:27:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:27:14.340550 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef451db-91d6-4cad-a6e3-5dbbfba1ccee None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:27:14.342136 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-9ef451db-91d6-4cad-a6e3-5dbbfba1ccee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:27:14.444597 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef451db-91d6-4cad-a6e3-5dbbfba1ccee None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:27:14.483230 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:27:14.517195 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2068feae-c1f4-44a3-8f24-566a676c14ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:27:14.517810 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2068feae-c1f4-44a3-8f24-566a676c14ce service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:27:14.518088 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2068feae-c1f4-44a3-8f24-566a676c14ce service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:27:14.518358 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2068feae-c1f4-44a3-8f24-566a676c14ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:27:14.519969 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-2068feae-c1f4-44a3-8f24-566a676c14ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:27:14.626760 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2068feae-c1f4-44a3-8f24-566a676c14ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:27:14.628195 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2068feae-c1f4-44a3-8f24-566a676c14ce service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:27:14.646728 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 719/1449] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:27:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:27:16.763792 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563c2aa2-69d9-4740-88ea-62f6f9d2c029 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:27:16.764364 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563c2aa2-69d9-4740-88ea-62f6f9d2c029 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:27:16.764364 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563c2aa2-69d9-4740-88ea-62f6f9d2c029 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:27:16.765762 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 731/1450] 127.0.0.1 () {54 vars in 917 bytes} [Sat Aug 6 01:27:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:27:16.771569 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9366ba89-8f88-4cc1-a84b-1604e261d658 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:27:16.772030 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9366ba89-8f88-4cc1-a84b-1604e261d658 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:27:16.772141 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9366ba89-8f88-4cc1-a84b-1604e261d658 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:27:16.839009 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-9366ba89-8f88-4cc1-a84b-1604e261d658 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:27:16.840244 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-9366ba89-8f88-4cc1-a84b-1604e261d658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:27:16.861085 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:27:16.936412 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 720/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Aug 6 01:27:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:27:16.976374 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563c2aa2-69d9-4740-88ea-62f6f9d2c029 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:27:16.977954 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-563c2aa2-69d9-4740-88ea-62f6f9d2c029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:27:17.082360 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563c2aa2-69d9-4740-88ea-62f6f9d2c029 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:27:17.102904 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:27:17.156574 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd04897e-ac8f-4670-bee5-78eeb919487b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:27:17.157236 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd04897e-ac8f-4670-bee5-78eeb919487b service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:27:17.157521 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd04897e-ac8f-4670-bee5-78eeb919487b service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:27:17.157795 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd04897e-ac8f-4670-bee5-78eeb919487b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:27:17.159407 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-dd04897e-ac8f-4670-bee5-78eeb919487b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:27:17.266115 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd04897e-ac8f-4670-bee5-78eeb919487b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:27:17.267546 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd04897e-ac8f-4670-bee5-78eeb919487b service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:27:17.285820 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 732/1452] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:27:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:27:19.345398 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb116ce8-d607-49e1-925f-b2ea5d5a89f4 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:27:19.345962 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb116ce8-d607-49e1-925f-b2ea5d5a89f4 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:27:19.345962 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb116ce8-d607-49e1-925f-b2ea5d5a89f4 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:27:19.347316 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 721/1453] 127.0.0.1 () {54 vars in 917 bytes} [Sat Aug 6 01:27:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:27:19.353170 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35be64df-92f9-4e3f-be17-45c01ff8f75f None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:27:19.353580 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35be64df-92f9-4e3f-be17-45c01ff8f75f None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:27:19.353755 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35be64df-92f9-4e3f-be17-45c01ff8f75f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:27:19.418175 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:27:19.420933 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-35be64df-92f9-4e3f-be17-45c01ff8f75f None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:27:19.422784 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-35be64df-92f9-4e3f-be17-45c01ff8f75f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:27:19.518687 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 733/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Aug 6 01:27:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:27:19.556930 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb116ce8-d607-49e1-925f-b2ea5d5a89f4 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:27:19.558483 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-bb116ce8-d607-49e1-925f-b2ea5d5a89f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:27:19.661579 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb116ce8-d607-49e1-925f-b2ea5d5a89f4 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:27:19.709654 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:27:19.734295 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38914c46-d9e8-4a6f-84cc-434ebe87c479 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:27:19.734914 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38914c46-d9e8-4a6f-84cc-434ebe87c479 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:27:19.735203 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38914c46-d9e8-4a6f-84cc-434ebe87c479 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:27:19.735478 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38914c46-d9e8-4a6f-84cc-434ebe87c479 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:27:19.737133 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-38914c46-d9e8-4a6f-84cc-434ebe87c479 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:27:19.844451 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38914c46-d9e8-4a6f-84cc-434ebe87c479 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:27:19.845912 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38914c46-d9e8-4a6f-84cc-434ebe87c479 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:27:19.864412 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 722/1455] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:27:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:27:21.936865 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584f8fa4-08c3-4fcc-9f42-a66ad7061899 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:27:21.937471 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584f8fa4-08c3-4fcc-9f42-a66ad7061899 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:27:21.937471 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584f8fa4-08c3-4fcc-9f42-a66ad7061899 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:27:21.938799 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 734/1456] 127.0.0.1 () {54 vars in 917 bytes} [Sat Aug 6 01:27:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:27:21.944206 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75b5a75-dc4c-4a74-a9cb-a7810e79c000 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:27:21.944635 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75b5a75-dc4c-4a74-a9cb-a7810e79c000 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:27:21.944766 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75b5a75-dc4c-4a74-a9cb-a7810e79c000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:27:22.010061 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-a75b5a75-dc4c-4a74-a9cb-a7810e79c000 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:27:22.011271 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-a75b5a75-dc4c-4a74-a9cb-a7810e79c000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:27:22.040312 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:27:22.108871 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 723/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Aug 6 01:27:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:27:22.147068 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584f8fa4-08c3-4fcc-9f42-a66ad7061899 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:27:22.148637 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-584f8fa4-08c3-4fcc-9f42-a66ad7061899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:27:22.253401 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584f8fa4-08c3-4fcc-9f42-a66ad7061899 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:27:22.282474 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:27:22.327227 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fc24a4-6c87-4d25-9e61-c801120373ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:27:22.327846 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fc24a4-6c87-4d25-9e61-c801120373ac service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:27:22.328131 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fc24a4-6c87-4d25-9e61-c801120373ac service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:27:22.328410 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fc24a4-6c87-4d25-9e61-c801120373ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:27:22.330080 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-68fc24a4-6c87-4d25-9e61-c801120373ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:27:22.559432 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fc24a4-6c87-4d25-9e61-c801120373ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:27:22.560755 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fc24a4-6c87-4d25-9e61-c801120373ac service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:27:22.578200 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 735/1458] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:27:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 01:27:24.626740 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca01829-b037-4e8e-8566-a0083ab3fb3a None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:27:24.627269 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca01829-b037-4e8e-8566-a0083ab3fb3a None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:27:24.627269 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca01829-b037-4e8e-8566-a0083ab3fb3a None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:27:24.628688 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 724/1459] 127.0.0.1 () {54 vars in 917 bytes} [Sat Aug 6 01:27:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:27:24.633955 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1bb023-b3b9-4dd6-a814-8507415af534 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:27:24.634385 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1bb023-b3b9-4dd6-a814-8507415af534 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:27:24.634502 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1bb023-b3b9-4dd6-a814-8507415af534 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:27:24.699898 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-0b1bb023-b3b9-4dd6-a814-8507415af534 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:27:24.701188 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-0b1bb023-b3b9-4dd6-a814-8507415af534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:27:24.709566 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:27:24.796857 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 736/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Aug 6 01:27:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:27:24.834884 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca01829-b037-4e8e-8566-a0083ab3fb3a None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:27:24.836782 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-1ca01829-b037-4e8e-8566-a0083ab3fb3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:27:24.940106 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca01829-b037-4e8e-8566-a0083ab3fb3a None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:27:24.994118 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:27:25.012777 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b175016e-8927-4309-a394-d2d196d16566 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:27:25.013433 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b175016e-8927-4309-a394-d2d196d16566 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:27:25.013719 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b175016e-8927-4309-a394-d2d196d16566 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:27:25.013996 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b175016e-8927-4309-a394-d2d196d16566 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:27:25.015649 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-b175016e-8927-4309-a394-d2d196d16566 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:27:25.125962 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b175016e-8927-4309-a394-d2d196d16566 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:27:25.127398 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b175016e-8927-4309-a394-d2d196d16566 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:27:25.145815 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 725/1461] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:27:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:27:27.167488 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe6e103-9a82-489d-a7d5-b14b7a84dac1 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:27:27.168074 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe6e103-9a82-489d-a7d5-b14b7a84dac1 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:27:27.168074 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe6e103-9a82-489d-a7d5-b14b7a84dac1 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:27:27.169449 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 737/1462] 127.0.0.1 () {54 vars in 917 bytes} [Sat Aug 6 01:27:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:27:27.174789 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406a8106-3687-4bf8-832f-d9d53ff62f9f None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:27:27.175232 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406a8106-3687-4bf8-832f-d9d53ff62f9f None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:27:27.175335 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406a8106-3687-4bf8-832f-d9d53ff62f9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:27:27.241763 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-406a8106-3687-4bf8-832f-d9d53ff62f9f None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:27:27.242984 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-406a8106-3687-4bf8-832f-d9d53ff62f9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:27:27.280652 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:27:27.338474 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 726/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Aug 6 01:27:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:27:27.377321 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe6e103-9a82-489d-a7d5-b14b7a84dac1 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:27:27.378836 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-bfe6e103-9a82-489d-a7d5-b14b7a84dac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:27:27.482202 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe6e103-9a82-489d-a7d5-b14b7a84dac1 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:27:27.519066 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:27:27.557727 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e14241-ffd4-4d71-aa48-b34f00c7de5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:27:27.558345 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e14241-ffd4-4d71-aa48-b34f00c7de5d service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:27:27.558626 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e14241-ffd4-4d71-aa48-b34f00c7de5d service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:27:27.558902 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e14241-ffd4-4d71-aa48-b34f00c7de5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:27:27.560520 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-58e14241-ffd4-4d71-aa48-b34f00c7de5d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:27:27.667599 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e14241-ffd4-4d71-aa48-b34f00c7de5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:27:27.668950 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e14241-ffd4-4d71-aa48-b34f00c7de5d service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:27:27.687737 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 738/1464] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:27:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:27:29.674269 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3585ec7d-fd7d-4485-8fb5-34e23173c948 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:27:29.674787 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3585ec7d-fd7d-4485-8fb5-34e23173c948 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:27:29.674787 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3585ec7d-fd7d-4485-8fb5-34e23173c948 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:27:29.676102 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 727/1465] 127.0.0.1 () {54 vars in 917 bytes} [Sat Aug 6 01:27:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:27:29.682227 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e85061-855b-4fc0-a212-9bf6935f559f None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:27:29.682667 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e85061-855b-4fc0-a212-9bf6935f559f None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:27:29.682778 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e85061-855b-4fc0-a212-9bf6935f559f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:27:29.747930 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-93e85061-855b-4fc0-a212-9bf6935f559f None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:27:29.749191 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-93e85061-855b-4fc0-a212-9bf6935f559f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:27:29.767264 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:27:29.844655 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 739/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Aug 6 01:27:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:27:29.883439 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3585ec7d-fd7d-4485-8fb5-34e23173c948 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:27:29.884979 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-3585ec7d-fd7d-4485-8fb5-34e23173c948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:27:29.988083 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3585ec7d-fd7d-4485-8fb5-34e23173c948 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:27:30.053488 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:27:30.060703 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d91e9a-7ea7-4128-824b-c682915d1699 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:27:30.061367 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d91e9a-7ea7-4128-824b-c682915d1699 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:27:30.061661 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d91e9a-7ea7-4128-824b-c682915d1699 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:27:30.061966 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d91e9a-7ea7-4128-824b-c682915d1699 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:27:30.063628 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-83d91e9a-7ea7-4128-824b-c682915d1699 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:27:30.171318 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d91e9a-7ea7-4128-824b-c682915d1699 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:27:30.172739 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d91e9a-7ea7-4128-824b-c682915d1699 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:27:30.191158 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 728/1467] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:27:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:27:32.184639 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583cf521-1ff6-4754-b1a7-b82d4655633d None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:27:32.185240 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583cf521-1ff6-4754-b1a7-b82d4655633d None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:27:32.185240 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583cf521-1ff6-4754-b1a7-b82d4655633d None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:27:32.186624 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 740/1468] 127.0.0.1 () {54 vars in 917 bytes} [Sat Aug 6 01:27:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:27:32.191771 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d66e57-d23c-462e-aafe-037413f1ee47 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:27:32.192732 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d66e57-d23c-462e-aafe-037413f1ee47 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:27:32.193254 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d66e57-d23c-462e-aafe-037413f1ee47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:27:32.259209 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-a4d66e57-d23c-462e-aafe-037413f1ee47 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:27:32.260417 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-a4d66e57-d23c-462e-aafe-037413f1ee47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:27:32.307974 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:27:32.357128 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 729/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Aug 6 01:27:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:27:32.394253 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583cf521-1ff6-4754-b1a7-b82d4655633d None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:27:32.396445 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-583cf521-1ff6-4754-b1a7-b82d4655633d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:27:32.499814 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583cf521-1ff6-4754-b1a7-b82d4655633d None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:27:32.547126 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:27:32.572520 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f487a7-d492-4664-ad43-b620d5b785b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:27:32.573165 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f487a7-d492-4664-ad43-b620d5b785b3 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:27:32.573450 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f487a7-d492-4664-ad43-b620d5b785b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:27:32.573724 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f487a7-d492-4664-ad43-b620d5b785b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:27:32.575343 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-45f487a7-d492-4664-ad43-b620d5b785b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:27:32.683308 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f487a7-d492-4664-ad43-b620d5b785b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:27:32.684748 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f487a7-d492-4664-ad43-b620d5b785b3 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:27:32.702834 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 741/1470] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:27:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:27:42.355005 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=22 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:27:42.700545 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=66 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:27:52.230047 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc787ce9-feee-42ff-b7d6-b36a3b86b0dc None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:27:52.230601 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc787ce9-feee-42ff-b7d6-b36a3b86b0dc None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:27:52.230601 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc787ce9-feee-42ff-b7d6-b36a3b86b0dc None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:27:52.231964 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 730/1471] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:27:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:27:52.237361 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca64be19-7bc3-4c7f-86d6-0a24d992c944 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:27:52.237756 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca64be19-7bc3-4c7f-86d6-0a24d992c944 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:27:52.237921 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca64be19-7bc3-4c7f-86d6-0a24d992c944 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:27:52.302693 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-ca64be19-7bc3-4c7f-86d6-0a24d992c944 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:27:52.303895 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-ca64be19-7bc3-4c7f-86d6-0a24d992c944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:27:52.399445 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 742/1472] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:27:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:27:52.482781 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe82656-e485-4786-99bd-b119e25214f2 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:27:52.483010 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe82656-e485-4786-99bd-b119e25214f2 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:27:52.483286 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe82656-e485-4786-99bd-b119e25214f2 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:27:52.484648 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 731/1473] 10.222.0.26 () {54 vars in 966 bytes} [Sat Aug 6 01:27:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:27:52.489624 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9583d4-0a80-413f-9b22-d2ff55d93b3d None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:27:52.489916 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9583d4-0a80-413f-9b22-d2ff55d93b3d None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:27:52.490198 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9583d4-0a80-413f-9b22-d2ff55d93b3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:27:52.564364 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-5c9583d4-0a80-413f-9b22-d2ff55d93b3d None None] MFA Rules not processed for user `abb76a5c8ef74bf2b18c56074daaad58`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:27:52.565596 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-5c9583d4-0a80-413f-9b22-d2ff55d93b3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:27:52.586779 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:27:52.641770 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 743/1474] 10.222.0.26 () {58 vars in 1063 bytes} [Sat Aug 6 01:27:52 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:27:52.650050 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe82656-e485-4786-99bd-b119e25214f2 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:27:52.651618 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-cbe82656-e485-4786-99bd-b119e25214f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:27:52.754134 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe82656-e485-4786-99bd-b119e25214f2 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:27:52.826471 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1f6f31-387f-40cb-b350-9fde61d199d0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abb76a5c8ef74bf2b18c56074daaad58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:27:52.827092 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1f6f31-387f-40cb-b350-9fde61d199d0 service glance] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:27:52.827377 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1f6f31-387f-40cb-b350-9fde61d199d0 service glance] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:27:52.827657 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1f6f31-387f-40cb-b350-9fde61d199d0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:27:52.829316 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-4c1f6f31-387f-40cb-b350-9fde61d199d0 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:27:52.884069 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:27:52.938473 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1f6f31-387f-40cb-b350-9fde61d199d0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:27:52.939917 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1f6f31-387f-40cb-b350-9fde61d199d0 service glance] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:27:52.959256 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 732/1475] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:27:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:27:55.063215 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58301c5-52b7-4d51-9be9-26bba835195b None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:27:55.063802 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58301c5-52b7-4d51-9be9-26bba835195b None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:27:55.063802 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58301c5-52b7-4d51-9be9-26bba835195b None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:27:55.065162 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 744/1476] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:27:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:27:55.070448 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6f7880-fa01-4a5c-9435-32fb451defa0 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:27:55.070886 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6f7880-fa01-4a5c-9435-32fb451defa0 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:27:55.070982 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6f7880-fa01-4a5c-9435-32fb451defa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:27:55.137224 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-6d6f7880-fa01-4a5c-9435-32fb451defa0 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:27:55.138425 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-6d6f7880-fa01-4a5c-9435-32fb451defa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:27:55.233919 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 733/1477] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:27:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:27:55.257068 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58301c5-52b7-4d51-9be9-26bba835195b None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:27:55.258592 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-e58301c5-52b7-4d51-9be9-26bba835195b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:27:55.382662 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58301c5-52b7-4d51-9be9-26bba835195b None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:27:55.431615 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:27:55.475975 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895239f5-b241-4b30-a48d-c7c758c12794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:27:55.476620 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895239f5-b241-4b30-a48d-c7c758c12794 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:27:55.476902 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895239f5-b241-4b30-a48d-c7c758c12794 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:27:55.477209 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895239f5-b241-4b30-a48d-c7c758c12794 admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:27:55.477908 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895239f5-b241-4b30-a48d-c7c758c12794 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:27:55.479656 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: WARNING py.warnings [None req-895239f5-b241-4b30-a48d-c7c758c12794 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 06 01:27:55.479656 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: warnings.warn(msg) Aug 06 01:27:55.479656 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: Aug 06 01:27:55.480794 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895239f5-b241-4b30-a48d-c7c758c12794 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:27:55.500051 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 745/1478] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Aug 6 01:27:55 2022] GET /identity/v3/projects => generated 2553 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:27:57.547268 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14debba-3c9f-4b6a-83b1-a71b1854af72 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:27:57.547887 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14debba-3c9f-4b6a-83b1-a71b1854af72 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:27:57.547887 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14debba-3c9f-4b6a-83b1-a71b1854af72 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:27:57.549192 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 734/1479] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:27:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:27:57.554544 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbcceea-55ea-4524-906d-f7d1285d5d16 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:27:57.554999 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbcceea-55ea-4524-906d-f7d1285d5d16 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:27:57.555156 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbcceea-55ea-4524-906d-f7d1285d5d16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:27:57.619914 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-afbcceea-55ea-4524-906d-f7d1285d5d16 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:27:57.621146 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-afbcceea-55ea-4524-906d-f7d1285d5d16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:27:57.714796 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 746/1480] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:27:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:27:57.758157 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14debba-3c9f-4b6a-83b1-a71b1854af72 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:27:57.759734 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-d14debba-3c9f-4b6a-83b1-a71b1854af72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:27:57.770483 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:27:57.906017 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14debba-3c9f-4b6a-83b1-a71b1854af72 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:27:57.990351 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:27:58.024855 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beab91b-2ad9-4510-a67d-9bb887bf2345 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87adc1d5bc240e99f95f6595b350be4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:27:58.025498 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beab91b-2ad9-4510-a67d-9bb887bf2345 service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:27:58.025780 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beab91b-2ad9-4510-a67d-9bb887bf2345 service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:27:58.026056 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beab91b-2ad9-4510-a67d-9bb887bf2345 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:27:58.027674 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-9beab91b-2ad9-4510-a67d-9bb887bf2345 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:27:58.116499 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9beab91b-2ad9-4510-a67d-9bb887bf2345 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:27:58.118142 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9beab91b-2ad9-4510-a67d-9bb887bf2345 service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:27:58.138534 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 735/1481] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:27:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:28:00.362281 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298ed53d-b1a3-4889-9975-45a7b572feeb None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:28:00.362819 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298ed53d-b1a3-4889-9975-45a7b572feeb None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:28:00.362819 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298ed53d-b1a3-4889-9975-45a7b572feeb None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:28:00.364283 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 747/1482] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:28:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:28:00.369641 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42b3e88-82ae-4e23-b562-7edcad8bd679 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:28:00.370060 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42b3e88-82ae-4e23-b562-7edcad8bd679 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:28:00.370197 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42b3e88-82ae-4e23-b562-7edcad8bd679 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:28:00.436185 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-d42b3e88-82ae-4e23-b562-7edcad8bd679 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:28:00.437439 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-d42b3e88-82ae-4e23-b562-7edcad8bd679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:28:00.484196 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:28:00.534543 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 736/1483] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:28:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:28:00.571961 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298ed53d-b1a3-4889-9975-45a7b572feeb None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:28:00.573545 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-298ed53d-b1a3-4889-9975-45a7b572feeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:28:00.597003 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:28:00.676652 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298ed53d-b1a3-4889-9975-45a7b572feeb None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:28:00.748732 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c2ba18-3e50-4753-b91f-de7afca7e5bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:28:00.749376 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c2ba18-3e50-4753-b91f-de7afca7e5bd service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:28:00.749659 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c2ba18-3e50-4753-b91f-de7afca7e5bd service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:28:00.749934 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c2ba18-3e50-4753-b91f-de7afca7e5bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:28:00.751541 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-63c2ba18-3e50-4753-b91f-de7afca7e5bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:28:00.826051 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:28:00.860914 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c2ba18-3e50-4753-b91f-de7afca7e5bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:28:00.862383 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c2ba18-3e50-4753-b91f-de7afca7e5bd service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:28:00.880562 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 748/1484] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:28:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:28:02.936080 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bfac39-d9ca-4d1a-91a2-53feada1f460 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:28:02.936644 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bfac39-d9ca-4d1a-91a2-53feada1f460 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:28:02.936644 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bfac39-d9ca-4d1a-91a2-53feada1f460 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:28:02.938018 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 737/1485] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:28:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:28:02.944387 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56d48fc-1134-4f0a-9d1a-57023f3f5860 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:28:02.944796 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56d48fc-1134-4f0a-9d1a-57023f3f5860 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:28:02.944937 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56d48fc-1134-4f0a-9d1a-57023f3f5860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:28:03.010447 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-e56d48fc-1134-4f0a-9d1a-57023f3f5860 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:28:03.011646 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-e56d48fc-1134-4f0a-9d1a-57023f3f5860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:28:03.106734 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 749/1486] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:28:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:28:03.148477 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bfac39-d9ca-4d1a-91a2-53feada1f460 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:28:03.150126 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-38bfac39-d9ca-4d1a-91a2-53feada1f460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:28:03.255479 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bfac39-d9ca-4d1a-91a2-53feada1f460 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:28:03.326461 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:28:03.328350 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d5c90f-4537-4dcd-996b-b5f5294c53e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:28:03.329104 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d5c90f-4537-4dcd-996b-b5f5294c53e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:28:03.329416 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d5c90f-4537-4dcd-996b-b5f5294c53e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:28:03.329763 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d5c90f-4537-4dcd-996b-b5f5294c53e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:28:03.331975 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-b2d5c90f-4537-4dcd-996b-b5f5294c53e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:28:03.441187 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d5c90f-4537-4dcd-996b-b5f5294c53e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:28:03.442618 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d5c90f-4537-4dcd-996b-b5f5294c53e9 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:28:03.461059 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 738/1487] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:28:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:28:05.628051 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a90bf68-39f0-45eb-b2da-14e403636a72 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:28:05.628607 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a90bf68-39f0-45eb-b2da-14e403636a72 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:28:05.628607 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a90bf68-39f0-45eb-b2da-14e403636a72 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:28:05.630597 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 750/1488] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:28:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:28:05.635951 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5259be-2003-4b40-b9e7-599b420ca8bf None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:28:05.636362 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5259be-2003-4b40-b9e7-599b420ca8bf None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:28:05.636514 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5259be-2003-4b40-b9e7-599b420ca8bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:28:05.701016 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-6c5259be-2003-4b40-b9e7-599b420ca8bf None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:28:05.702230 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-6c5259be-2003-4b40-b9e7-599b420ca8bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:28:05.753666 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:28:05.797785 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 739/1489] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:28:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:28:05.839055 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a90bf68-39f0-45eb-b2da-14e403636a72 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:28:05.840585 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-9a90bf68-39f0-45eb-b2da-14e403636a72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:28:05.877974 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:28:05.947484 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a90bf68-39f0-45eb-b2da-14e403636a72 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:28:06.019229 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd84ed23-e037-4d63-90ab-b560b46a6699 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:28:06.019842 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd84ed23-e037-4d63-90ab-b560b46a6699 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:28:06.020159 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd84ed23-e037-4d63-90ab-b560b46a6699 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:28:06.020391 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd84ed23-e037-4d63-90ab-b560b46a6699 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:28:06.022015 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-fd84ed23-e037-4d63-90ab-b560b46a6699 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:28:06.099880 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:28:06.131278 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd84ed23-e037-4d63-90ab-b560b46a6699 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:28:06.133995 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd84ed23-e037-4d63-90ab-b560b46a6699 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:28:06.154574 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 751/1490] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:28:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:28:08.385927 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e37f04-6f5c-4f4d-83dd-b7ef9bbe8c0d None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:28:08.386456 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e37f04-6f5c-4f4d-83dd-b7ef9bbe8c0d None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:28:08.386456 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e37f04-6f5c-4f4d-83dd-b7ef9bbe8c0d None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:28:08.387840 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 740/1491] 10.222.0.26 () {54 vars in 927 bytes} [Sat Aug 6 01:28:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:28:08.393004 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8122e917-f9a5-45c5-8d9a-fdaadfcfc2d2 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:28:08.393418 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8122e917-f9a5-45c5-8d9a-fdaadfcfc2d2 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:28:08.393601 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8122e917-f9a5-45c5-8d9a-fdaadfcfc2d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:28:08.459570 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-8122e917-f9a5-45c5-8d9a-fdaadfcfc2d2 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:28:08.460766 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-8122e917-f9a5-45c5-8d9a-fdaadfcfc2d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:28:08.556952 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 752/1492] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Aug 6 01:28:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:28:08.597125 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e37f04-6f5c-4f4d-83dd-b7ef9bbe8c0d None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:28:08.598895 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-50e37f04-6f5c-4f4d-83dd-b7ef9bbe8c0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:28:08.701840 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e37f04-6f5c-4f4d-83dd-b7ef9bbe8c0d None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:28:08.773535 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9483bdf-b8cc-4681-9d26-0f3b2aa004b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:28:08.774147 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9483bdf-b8cc-4681-9d26-0f3b2aa004b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:28:08.774425 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9483bdf-b8cc-4681-9d26-0f3b2aa004b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:28:08.774698 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9483bdf-b8cc-4681-9d26-0f3b2aa004b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:28:08.776299 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-a9483bdf-b8cc-4681-9d26-0f3b2aa004b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:28:08.785650 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:28:08.885622 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9483bdf-b8cc-4681-9d26-0f3b2aa004b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:28:08.887062 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9483bdf-b8cc-4681-9d26-0f3b2aa004b0 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:28:08.904839 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 741/1493] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:28:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:28:18.555217 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=82 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:28:18.902544 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=50 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:29:18.839763 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aab554-a23f-4205-afcf-5342ca5a9f6f None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:29:18.840355 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aab554-a23f-4205-afcf-5342ca5a9f6f None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:29:18.840355 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aab554-a23f-4205-afcf-5342ca5a9f6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:29:18.913892 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-36aab554-a23f-4205-afcf-5342ca5a9f6f None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:29:18.915088 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-36aab554-a23f-4205-afcf-5342ca5a9f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:29:19.010510 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 753/1494] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:29:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:29:19.018719 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfda27c-7c2a-4ee6-8c9d-3c265e6da2a4 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:29:19.019208 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfda27c-7c2a-4ee6-8c9d-3c265e6da2a4 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:29:19.019380 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfda27c-7c2a-4ee6-8c9d-3c265e6da2a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:29:19.094214 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-8cfda27c-7c2a-4ee6-8c9d-3c265e6da2a4 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:29:19.095326 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-8cfda27c-7c2a-4ee6-8c9d-3c265e6da2a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:29:19.190704 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 742/1495] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:29:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:29:19.196273 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36aab554-a23f-4205-afcf-5342ca5a9f6f None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:29:19.197507 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-36aab554-a23f-4205-afcf-5342ca5a9f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:29:19.208815 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36aab554-a23f-4205-afcf-5342ca5a9f6f None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:29:19.271128 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:29:19.306154 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91543656-53df-44ef-be8f-f4e35ee1ebd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:29:19.306778 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91543656-53df-44ef-be8f-f4e35ee1ebd1 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:29:19.307060 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91543656-53df-44ef-be8f-f4e35ee1ebd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:29:19.307332 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91543656-53df-44ef-be8f-f4e35ee1ebd1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:29:19.308025 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91543656-53df-44ef-be8f-f4e35ee1ebd1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:29:19.308332 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91543656-53df-44ef-be8f-f4e35ee1ebd1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:29:19.309385 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: WARNING py.warnings [None req-91543656-53df-44ef-be8f-f4e35ee1ebd1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 06 01:29:19.309385 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: warnings.warn(msg) Aug 06 01:29:19.309385 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: Aug 06 01:29:19.310338 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91543656-53df-44ef-be8f-f4e35ee1ebd1 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:29:19.321485 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 754/1496] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Aug 6 01:29:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:29:19.330053 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfda27c-7c2a-4ee6-8c9d-3c265e6da2a4 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:29:19.339378 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfda27c-7c2a-4ee6-8c9d-3c265e6da2a4 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:29:19.400397 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:29:19.436238 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708e1c17-9b11-40a8-9845-16fc242bdca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:29:19.436853 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708e1c17-9b11-40a8-9845-16fc242bdca0 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:29:19.437162 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708e1c17-9b11-40a8-9845-16fc242bdca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:29:19.437437 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708e1c17-9b11-40a8-9845-16fc242bdca0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:29:19.438098 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708e1c17-9b11-40a8-9845-16fc242bdca0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:29:19.439807 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: WARNING py.warnings [None req-708e1c17-9b11-40a8-9845-16fc242bdca0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 06 01:29:19.439807 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: warnings.warn(msg) Aug 06 01:29:19.439807 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: Aug 06 01:29:19.440998 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708e1c17-9b11-40a8-9845-16fc242bdca0 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:29:19.496553 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 743/1497] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Aug 6 01:29:19 2022] POST /identity/v3/projects => generated 377 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:29:19.500871 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91543656-53df-44ef-be8f-f4e35ee1ebd1 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:29:19.502439 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-91543656-53df-44ef-be8f-f4e35ee1ebd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:29:19.536202 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91543656-53df-44ef-be8f-f4e35ee1ebd1 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:29:19.627694 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ca26f4-e0e2-4e49-9fe5-5e44c655437e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:29:19.628310 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ca26f4-e0e2-4e49-9fe5-5e44c655437e admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:29:19.628592 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ca26f4-e0e2-4e49-9fe5-5e44c655437e admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:29:19.628890 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ca26f4-e0e2-4e49-9fe5-5e44c655437e admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:29:19.629589 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ca26f4-e0e2-4e49-9fe5-5e44c655437e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:29:19.630606 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: WARNING py.warnings [None req-17ca26f4-e0e2-4e49-9fe5-5e44c655437e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 06 01:29:19.630606 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: warnings.warn(msg) Aug 06 01:29:19.630606 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: Aug 06 01:29:19.631806 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ca26f4-e0e2-4e49-9fe5-5e44c655437e admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:29:19.673006 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 755/1498] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:29:19 2022] POST /identity/v3/users => generated 289 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:29:19.677131 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708e1c17-9b11-40a8-9845-16fc242bdca0 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:29:19.678649 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-708e1c17-9b11-40a8-9845-16fc242bdca0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:29:19.740724 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708e1c17-9b11-40a8-9845-16fc242bdca0 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:29:19.821428 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:29:19.833399 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acec984-3ae8-4a7a-b565-5911203a6e00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:29:19.834012 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acec984-3ae8-4a7a-b565-5911203a6e00 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:29:19.834289 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acec984-3ae8-4a7a-b565-5911203a6e00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:29:19.834561 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acec984-3ae8-4a7a-b565-5911203a6e00 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:29:19.835246 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acec984-3ae8-4a7a-b565-5911203a6e00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:29:19.836240 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: WARNING py.warnings [None req-3acec984-3ae8-4a7a-b565-5911203a6e00 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 06 01:29:19.836240 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: warnings.warn(msg) Aug 06 01:29:19.836240 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: Aug 06 01:29:19.837146 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acec984-3ae8-4a7a-b565-5911203a6e00 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:29:19.849913 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 744/1499] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:29:19 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:29:19.854038 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ca26f4-e0e2-4e49-9fe5-5e44c655437e admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:29:19.855570 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-17ca26f4-e0e2-4e49-9fe5-5e44c655437e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:29:19.867314 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ca26f4-e0e2-4e49-9fe5-5e44c655437e admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:29:19.891801 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:29:19.963929 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8df261e-4a19-4210-be61-3c8fc0953020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:29:19.964548 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8df261e-4a19-4210-be61-3c8fc0953020 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:29:19.964836 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8df261e-4a19-4210-be61-3c8fc0953020 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:29:19.965136 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8df261e-4a19-4210-be61-3c8fc0953020 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:29:19.965830 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8df261e-4a19-4210-be61-3c8fc0953020 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:29:19.966845 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: WARNING py.warnings [None req-d8df261e-4a19-4210-be61-3c8fc0953020 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 06 01:29:19.966845 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: warnings.warn(msg) Aug 06 01:29:19.966845 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: Aug 06 01:29:19.967729 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8df261e-4a19-4210-be61-3c8fc0953020 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:29:19.980361 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 756/1500] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:29:19 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:29:19.984934 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acec984-3ae8-4a7a-b565-5911203a6e00 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:29:19.986521 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-3acec984-3ae8-4a7a-b565-5911203a6e00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:29:20.020785 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acec984-3ae8-4a7a-b565-5911203a6e00 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:29:20.113113 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5cf905-1be9-4a30-a911-7578aebe446b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:29:20.114030 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5cf905-1be9-4a30-a911-7578aebe446b admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:29:20.114313 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5cf905-1be9-4a30-a911-7578aebe446b admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:29:20.114589 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5cf905-1be9-4a30-a911-7578aebe446b admin admin] PATH_INFO: `/v3/projects/0f3cc0ee3f4748aaab727da11b27c584/users/056a85ddbf3e4c8894e6384e526c31df/roles/61b3b93d37a047e4bb8369b46fe20db9` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:29:20.136509 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5cf905-1be9-4a30-a911-7578aebe446b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f3cc0ee3f4748aaab727da11b27c584, user_id=056a85ddbf3e4c8894e6384e526c31df, role_id=61b3b93d37a047e4bb8369b46fe20db9)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:29:20.137600 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: WARNING py.warnings [None req-7a5cf905-1be9-4a30-a911-7578aebe446b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 06 01:29:20.137600 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: warnings.warn(msg) Aug 06 01:29:20.137600 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: Aug 06 01:29:20.139558 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5cf905-1be9-4a30-a911-7578aebe446b admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:29:20.163976 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 745/1501] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:29:19 2022] PUT /identity/v3/projects/0f3cc0ee3f4748aaab727da11b27c584/users/056a85ddbf3e4c8894e6384e526c31df/roles/61b3b93d37a047e4bb8369b46fe20db9 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:29:20.176513 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b21100-9f15-4951-ac37-a6c2394bf83b None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:29:20.176811 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b21100-9f15-4951-ac37-a6c2394bf83b None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:29:20.177123 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b21100-9f15-4951-ac37-a6c2394bf83b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:29:20.233678 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-65b21100-9f15-4951-ac37-a6c2394bf83b None None] MFA Rules not processed for user `056a85ddbf3e4c8894e6384e526c31df`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:29:20.235079 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-65b21100-9f15-4951-ac37-a6c2394bf83b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:29:20.310929 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 757/1502] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:29:20 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:29:20.324612 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c0b34a-22fb-4bfd-b978-0b208c5d9018 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:29:20.324909 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c0b34a-22fb-4bfd-b978-0b208c5d9018 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:29:20.325227 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c0b34a-22fb-4bfd-b978-0b208c5d9018 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:29:20.326587 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 746/1503] 10.222.0.26 () {54 vars in 964 bytes} [Sat Aug 6 01:29:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:29:20.331391 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345b46c4-008e-4207-8926-7fae6a72c93c None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:29:20.331779 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345b46c4-008e-4207-8926-7fae6a72c93c None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:29:20.332583 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345b46c4-008e-4207-8926-7fae6a72c93c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:29:20.362891 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:29:20.410169 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-345b46c4-008e-4207-8926-7fae6a72c93c None None] MFA Rules not processed for user `1be1e6173e374a60b9e1fbf51a1ab571`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:29:20.411368 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-345b46c4-008e-4207-8926-7fae6a72c93c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:29:20.486219 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 758/1504] 10.222.0.26 () {58 vars in 1061 bytes} [Sat Aug 6 01:29:20 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:29:20.494396 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c0b34a-22fb-4bfd-b978-0b208c5d9018 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:29:20.495912 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-08c0b34a-22fb-4bfd-b978-0b208c5d9018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:29:20.545700 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:29:20.600678 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c0b34a-22fb-4bfd-b978-0b208c5d9018 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:29:20.673888 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ba7bf5-97a1-474c-b11c-3d3e8612e23c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1be1e6173e374a60b9e1fbf51a1ab571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:29:20.674501 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ba7bf5-97a1-474c-b11c-3d3e8612e23c service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:29:20.674787 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ba7bf5-97a1-474c-b11c-3d3e8612e23c service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:29:20.675064 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ba7bf5-97a1-474c-b11c-3d3e8612e23c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:29:20.677226 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-58ba7bf5-97a1-474c-b11c-3d3e8612e23c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:29:20.775228 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ba7bf5-97a1-474c-b11c-3d3e8612e23c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:29:20.776585 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ba7bf5-97a1-474c-b11c-3d3e8612e23c service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:29:20.780089 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:29:20.796361 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 747/1505] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:29:20 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:29:20.842035 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b900dd-dd4a-4589-b1b0-a52065fac0c6 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:29:20.842339 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b900dd-dd4a-4589-b1b0-a52065fac0c6 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:29:20.842650 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b900dd-dd4a-4589-b1b0-a52065fac0c6 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:29:20.843921 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 759/1506] 10.222.0.26 () {52 vars in 830 bytes} [Sat Aug 6 01:29:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:29:20.847647 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ba7bf5-97a1-474c-b11c-3d3e8612e23c service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:29:20.848863 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-58ba7bf5-97a1-474c-b11c-3d3e8612e23c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:29:20.946533 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ba7bf5-97a1-474c-b11c-3d3e8612e23c service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:29:21.043146 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46672c40-7e35-451b-93c5-7ac20d03c815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:29:21.043787 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46672c40-7e35-451b-93c5-7ac20d03c815 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:29:21.044094 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46672c40-7e35-451b-93c5-7ac20d03c815 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:29:21.044367 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46672c40-7e35-451b-93c5-7ac20d03c815 admin admin] PATH_INFO: `/v3/users/056a85ddbf3e4c8894e6384e526c31df` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:29:21.056586 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46672c40-7e35-451b-93c5-7ac20d03c815 admin admin] RBAC: Authorizing `identity:delete_user(user_id=056a85ddbf3e4c8894e6384e526c31df)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:29:21.057644 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: WARNING py.warnings [None req-46672c40-7e35-451b-93c5-7ac20d03c815 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 06 01:29:21.057644 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: warnings.warn(msg) Aug 06 01:29:21.057644 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: Aug 06 01:29:21.058856 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46672c40-7e35-451b-93c5-7ac20d03c815 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:29:21.068848 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:29:21.126527 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-46672c40-7e35-451b-93c5-7ac20d03c815 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '056a85ddbf3e4c8894e6384e526c31df'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:29:21.127063 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-46672c40-7e35-451b-93c5-7ac20d03c815 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '056a85ddbf3e4c8894e6384e526c31df'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:29:21.127492 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-46672c40-7e35-451b-93c5-7ac20d03c815 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '056a85ddbf3e4c8894e6384e526c31df'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:29:21.151350 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-46672c40-7e35-451b-93c5-7ac20d03c815 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '056a85ddbf3e4c8894e6384e526c31df'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:29:21.165007 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-46672c40-7e35-451b-93c5-7ac20d03c815 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '056a85ddbf3e4c8894e6384e526c31df'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:29:21.179909 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 748/1507] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Aug 6 01:29:20 2022] DELETE /identity/v3/users/056a85ddbf3e4c8894e6384e526c31df => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:29:21.184222 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4783f33c-329c-46a7-bb04-50cd6d2b48af None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:29:21.184674 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4783f33c-329c-46a7-bb04-50cd6d2b48af None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:29:21.184803 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4783f33c-329c-46a7-bb04-50cd6d2b48af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:29:21.241703 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-4783f33c-329c-46a7-bb04-50cd6d2b48af None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:29:21.242911 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-4783f33c-329c-46a7-bb04-50cd6d2b48af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:29:21.317224 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:29:21.331979 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 760/1508] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:29:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:29:21.342150 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46672c40-7e35-451b-93c5-7ac20d03c815 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:29:21.343391 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-46672c40-7e35-451b-93c5-7ac20d03c815 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:29:21.444566 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46672c40-7e35-451b-93c5-7ac20d03c815 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:29:21.497887 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:29:21.519655 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aacd67-02d0-4b90-a1a8-0c17f1c0d36e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:29:21.520412 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aacd67-02d0-4b90-a1a8-0c17f1c0d36e service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:29:21.520705 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aacd67-02d0-4b90-a1a8-0c17f1c0d36e service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:29:21.520985 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aacd67-02d0-4b90-a1a8-0c17f1c0d36e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:29:21.522687 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-92aacd67-02d0-4b90-a1a8-0c17f1c0d36e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:29:21.632706 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92aacd67-02d0-4b90-a1a8-0c17f1c0d36e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:29:21.634109 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92aacd67-02d0-4b90-a1a8-0c17f1c0d36e service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:29:21.652160 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 749/1509] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:29:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:29:21.981611 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4783f33c-329c-46a7-bb04-50cd6d2b48af None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:29:21.982845 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-4783f33c-329c-46a7-bb04-50cd6d2b48af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:29:21.995073 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4783f33c-329c-46a7-bb04-50cd6d2b48af None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:29:22.093383 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b4e46d-2177-4771-95c0-5e1718000140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:29:22.094029 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b4e46d-2177-4771-95c0-5e1718000140 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:29:22.094318 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b4e46d-2177-4771-95c0-5e1718000140 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:29:22.094590 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b4e46d-2177-4771-95c0-5e1718000140 admin admin] PATH_INFO: `/v3/projects/0f3cc0ee3f4748aaab727da11b27c584` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:29:22.105001 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b4e46d-2177-4771-95c0-5e1718000140 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f3cc0ee3f4748aaab727da11b27c584)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:29:22.106057 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: WARNING py.warnings [None req-66b4e46d-2177-4771-95c0-5e1718000140 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 06 01:29:22.106057 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: warnings.warn(msg) Aug 06 01:29:22.106057 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: Aug 06 01:29:22.107259 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b4e46d-2177-4771-95c0-5e1718000140 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:29:22.165321 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-66b4e46d-2177-4771-95c0-5e1718000140 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f3cc0ee3f4748aaab727da11b27c584', 'request_id': 'req-66b4e46d-2177-4771-95c0-5e1718000140'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:29:22.178054 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-66b4e46d-2177-4771-95c0-5e1718000140 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f3cc0ee3f4748aaab727da11b27c584', 'request_id': 'req-66b4e46d-2177-4771-95c0-5e1718000140'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:29:22.183952 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-66b4e46d-2177-4771-95c0-5e1718000140 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f3cc0ee3f4748aaab727da11b27c584', 'request_id': 'req-66b4e46d-2177-4771-95c0-5e1718000140'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:29:22.210152 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-66b4e46d-2177-4771-95c0-5e1718000140 admin admin] The token cache is being invalidate because project 0f3cc0ee3f4748aaab727da11b27c584 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:29:22.210831 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-66b4e46d-2177-4771-95c0-5e1718000140 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:29:22.210966 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-66b4e46d-2177-4771-95c0-5e1718000140 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:29:22.212297 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 761/1510] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Aug 6 01:29:21 2022] DELETE /identity/v3/projects/0f3cc0ee3f4748aaab727da11b27c584 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:29:31.649896 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=62 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:29:32.208590 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:30:48.893691 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1188bd8-9f7c-46b2-804b-e326854704eb None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:48.894288 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1188bd8-9f7c-46b2-804b-e326854704eb None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:48.894288 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1188bd8-9f7c-46b2-804b-e326854704eb None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:48.895613 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 750/1511] 127.0.0.1 () {54 vars in 901 bytes} [Sat Aug 6 01:30:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:30:48.900517 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d87549-f467-48d0-aaa4-a4b7f9ec6435 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:48.900979 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d87549-f467-48d0-aaa4-a4b7f9ec6435 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:48.901114 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d87549-f467-48d0-aaa4-a4b7f9ec6435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:48.982172 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-d0d87549-f467-48d0-aaa4-a4b7f9ec6435 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:30:48.983376 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-d0d87549-f467-48d0-aaa4-a4b7f9ec6435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:30:49.087937 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 762/1512] 127.0.0.1 () {58 vars in 998 bytes} [Sat Aug 6 01:30:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:30:49.245832 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1188bd8-9f7c-46b2-804b-e326854704eb None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:49.248063 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-b1188bd8-9f7c-46b2-804b-e326854704eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:30:49.365554 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1188bd8-9f7c-46b2-804b-e326854704eb None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:49.371048 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43519269-cb9f-4b71-9eeb-01f7b0d6d3b8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87adc1d5bc240e99f95f6595b350be4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:49.371665 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43519269-cb9f-4b71-9eeb-01f7b0d6d3b8 service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:49.371947 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43519269-cb9f-4b71-9eeb-01f7b0d6d3b8 service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:49.372227 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43519269-cb9f-4b71-9eeb-01f7b0d6d3b8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:49.380220 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43519269-cb9f-4b71-9eeb-01f7b0d6d3b8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:49.381963 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43519269-cb9f-4b71-9eeb-01f7b0d6d3b8 service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:49.391490 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 751/1513] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:30:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:30:50.227548 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f754f807-7a2c-4f69-8ca7-75e90909da25 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:50.228084 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f754f807-7a2c-4f69-8ca7-75e90909da25 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:50.228084 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f754f807-7a2c-4f69-8ca7-75e90909da25 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:50.229538 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 763/1514] 127.0.0.1 () {54 vars in 901 bytes} [Sat Aug 6 01:30:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:30:50.234354 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d60d9e-a2e9-45a3-bbff-97c7fbe158a2 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:50.234779 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d60d9e-a2e9-45a3-bbff-97c7fbe158a2 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:50.234914 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d60d9e-a2e9-45a3-bbff-97c7fbe158a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:50.254906 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-34d60d9e-a2e9-45a3-bbff-97c7fbe158a2 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:30:50.256862 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-34d60d9e-a2e9-45a3-bbff-97c7fbe158a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:30:50.264675 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 752/1515] 127.0.0.1 () {58 vars in 998 bytes} [Sat Aug 6 01:30:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:30:50.414633 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f754f807-7a2c-4f69-8ca7-75e90909da25 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:50.429309 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f754f807-7a2c-4f69-8ca7-75e90909da25 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:50.434754 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2de819d-a0fc-4773-a9ad-18b318e5d2dc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87adc1d5bc240e99f95f6595b350be4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:50.435360 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2de819d-a0fc-4773-a9ad-18b318e5d2dc service nova] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:50.435639 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2de819d-a0fc-4773-a9ad-18b318e5d2dc service nova] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:50.435911 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2de819d-a0fc-4773-a9ad-18b318e5d2dc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:50.444822 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2de819d-a0fc-4773-a9ad-18b318e5d2dc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:50.446422 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2de819d-a0fc-4773-a9ad-18b318e5d2dc service nova] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:50.455756 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 764/1516] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:30:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:30:51.056490 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178ff4a3-98c6-49d2-ac4e-f6a3c172b948 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:51.056781 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178ff4a3-98c6-49d2-ac4e-f6a3c172b948 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:51.057111 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178ff4a3-98c6-49d2-ac4e-f6a3c172b948 None None] PATH_INFO: `/v3` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:51.058447 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 753/1517] 10.222.0.33 () {54 vars in 925 bytes} [Sat Aug 6 01:30:51 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:30:51.065237 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62a5674-1ce9-48b2-94fd-22fbf3bceb56 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:51.065528 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62a5674-1ce9-48b2-94fd-22fbf3bceb56 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:51.065802 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62a5674-1ce9-48b2-94fd-22fbf3bceb56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:51.126293 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-d62a5674-1ce9-48b2-94fd-22fbf3bceb56 None None] MFA Rules not processed for user `fa3b93ddd9464a2cba724d67b56b477a`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:30:51.127504 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-d62a5674-1ce9-48b2-94fd-22fbf3bceb56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:30:51.144177 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:30:51.198276 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 765/1518] 10.222.0.33 () {58 vars in 1015 bytes} [Sat Aug 6 01:30:51 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:31:00.181234 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d391bc33-74a3-4508-bd76-09fc80ef00d8 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:31:00.181769 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d391bc33-74a3-4508-bd76-09fc80ef00d8 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:31:00.181769 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d391bc33-74a3-4508-bd76-09fc80ef00d8 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:31:00.183077 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 754/1519] 10.222.0.26 () {54 vars in 961 bytes} [Sat Aug 6 01:31:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:31:00.187915 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56db38b9-45c8-4e34-915e-89551cd748c7 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:31:00.188298 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56db38b9-45c8-4e34-915e-89551cd748c7 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:31:00.188478 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56db38b9-45c8-4e34-915e-89551cd748c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:31:00.207999 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-56db38b9-45c8-4e34-915e-89551cd748c7 None None] MFA Rules not processed for user `fa3b93ddd9464a2cba724d67b56b477a`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:31:00.209793 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-56db38b9-45c8-4e34-915e-89551cd748c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:31:00.218283 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 766/1520] 10.222.0.26 () {58 vars in 1058 bytes} [Sat Aug 6 01:31:00 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:31:00.225812 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d391bc33-74a3-4508-bd76-09fc80ef00d8 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:31:00.239491 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d391bc33-74a3-4508-bd76-09fc80ef00d8 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:31:00.244509 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9d9505-160e-4dd5-9e5d-cf0b725b980f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa3b93ddd9464a2cba724d67b56b477a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:31:00.245146 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9d9505-160e-4dd5-9e5d-cf0b725b980f service placement] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:31:00.245431 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9d9505-160e-4dd5-9e5d-cf0b725b980f service placement] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:31:00.245707 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9d9505-160e-4dd5-9e5d-cf0b725b980f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:31:00.253880 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9d9505-160e-4dd5-9e5d-cf0b725b980f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:31:00.255961 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9d9505-160e-4dd5-9e5d-cf0b725b980f service placement] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:31:00.265544 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 755/1521] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:31:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:31:01.680908 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd96ac2-33a0-4abd-83a5-942067d3eb20 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:31:01.681553 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd96ac2-33a0-4abd-83a5-942067d3eb20 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:31:01.681553 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd96ac2-33a0-4abd-83a5-942067d3eb20 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:31:01.682799 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 767/1522] 127.0.0.1 () {54 vars in 901 bytes} [Sat Aug 6 01:31:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:31:01.687701 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83602ff-8cbc-47de-98d1-199d78d0ed6a None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:31:01.688137 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83602ff-8cbc-47de-98d1-199d78d0ed6a None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:31:01.688269 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83602ff-8cbc-47de-98d1-199d78d0ed6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:31:01.707725 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-d83602ff-8cbc-47de-98d1-199d78d0ed6a None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:31:01.709576 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-d83602ff-8cbc-47de-98d1-199d78d0ed6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:31:01.717433 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 756/1523] 127.0.0.1 () {58 vars in 998 bytes} [Sat Aug 6 01:31:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:31:01.875166 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd96ac2-33a0-4abd-83a5-942067d3eb20 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:31:01.890488 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd96ac2-33a0-4abd-83a5-942067d3eb20 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:31:01.895970 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f039a170-5064-4390-9496-811fa002b48f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87adc1d5bc240e99f95f6595b350be4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:31:01.897389 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f039a170-5064-4390-9496-811fa002b48f service nova] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:31:01.897719 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f039a170-5064-4390-9496-811fa002b48f service nova] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:31:01.897998 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f039a170-5064-4390-9496-811fa002b48f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:31:01.906064 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f039a170-5064-4390-9496-811fa002b48f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:31:01.907648 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f039a170-5064-4390-9496-811fa002b48f service nova] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:31:01.917342 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 768/1524] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:31:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:31:02.734388 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e87e4f-7cbd-4139-8364-47a1019e5dd3 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:31:02.734938 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e87e4f-7cbd-4139-8364-47a1019e5dd3 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:31:02.734938 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e87e4f-7cbd-4139-8364-47a1019e5dd3 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:31:02.736295 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 757/1525] 127.0.0.1 () {54 vars in 901 bytes} [Sat Aug 6 01:31:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:31:02.741950 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b2a62d-8ff4-42dc-b310-2ad70bb1df54 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:31:02.742354 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b2a62d-8ff4-42dc-b310-2ad70bb1df54 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:31:02.742507 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b2a62d-8ff4-42dc-b310-2ad70bb1df54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:31:02.761439 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-08b2a62d-8ff4-42dc-b310-2ad70bb1df54 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:31:02.762638 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-08b2a62d-8ff4-42dc-b310-2ad70bb1df54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:31:02.770400 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 769/1526] 127.0.0.1 () {58 vars in 998 bytes} [Sat Aug 6 01:31:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:31:02.922759 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e87e4f-7cbd-4139-8364-47a1019e5dd3 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:31:02.937303 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e87e4f-7cbd-4139-8364-47a1019e5dd3 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:31:02.942766 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c909b8-84a2-4610-af47-657822a47994 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87adc1d5bc240e99f95f6595b350be4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:31:02.943524 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c909b8-84a2-4610-af47-657822a47994 service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:31:02.943750 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c909b8-84a2-4610-af47-657822a47994 service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:31:02.944037 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c909b8-84a2-4610-af47-657822a47994 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:31:02.952062 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c909b8-84a2-4610-af47-657822a47994 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:31:02.953676 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c909b8-84a2-4610-af47-657822a47994 service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:31:02.963872 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 758/1527] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:31:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:31:06.686779 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a82391-6454-4927-8e23-338bb8dd2927 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:31:06.687318 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a82391-6454-4927-8e23-338bb8dd2927 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:31:06.687318 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a82391-6454-4927-8e23-338bb8dd2927 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:31:06.688663 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 770/1528] 127.0.0.1 () {54 vars in 917 bytes} [Sat Aug 6 01:31:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:31:06.694060 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b99606-1648-4fca-958a-1f6075c3018a None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:31:06.694470 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b99606-1648-4fca-958a-1f6075c3018a None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:31:06.694619 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b99606-1648-4fca-958a-1f6075c3018a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:31:06.714873 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-17b99606-1648-4fca-958a-1f6075c3018a None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:31:06.716110 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-17b99606-1648-4fca-958a-1f6075c3018a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:31:06.723971 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 759/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Aug 6 01:31:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:31:06.797792 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a82391-6454-4927-8e23-338bb8dd2927 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:31:06.799627 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-14a82391-6454-4927-8e23-338bb8dd2927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:31:06.876519 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a82391-6454-4927-8e23-338bb8dd2927 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:31:06.880033 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefdf8fd-35a8-4a61-b6cc-baa930a64991 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:31:06.881259 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefdf8fd-35a8-4a61-b6cc-baa930a64991 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:31:06.881547 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefdf8fd-35a8-4a61-b6cc-baa930a64991 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:31:06.881824 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefdf8fd-35a8-4a61-b6cc-baa930a64991 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:31:06.890648 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefdf8fd-35a8-4a61-b6cc-baa930a64991 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:31:06.892075 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefdf8fd-35a8-4a61-b6cc-baa930a64991 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:31:06.902071 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 771/1530] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:31:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:31:08.880091 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b15285-fee6-4c1f-8e9a-1d1746e64268 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:31:08.881290 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b15285-fee6-4c1f-8e9a-1d1746e64268 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:31:08.881760 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b15285-fee6-4c1f-8e9a-1d1746e64268 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:31:08.883130 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 760/1531] 127.0.0.1 () {54 vars in 917 bytes} [Sat Aug 6 01:31:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:31:08.888325 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9030153-317a-4a21-89eb-e213c3916de6 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:31:08.888701 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9030153-317a-4a21-89eb-e213c3916de6 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:31:08.888882 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9030153-317a-4a21-89eb-e213c3916de6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:31:08.909073 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-f9030153-317a-4a21-89eb-e213c3916de6 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:31:08.910274 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-f9030153-317a-4a21-89eb-e213c3916de6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:31:08.918153 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 772/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Aug 6 01:31:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:31:08.955360 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b15285-fee6-4c1f-8e9a-1d1746e64268 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:31:08.966399 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b15285-fee6-4c1f-8e9a-1d1746e64268 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:31:08.969914 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5cd89e-29d4-49ab-8a0a-c6ddd10dfa8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:31:08.970516 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5cd89e-29d4-49ab-8a0a-c6ddd10dfa8b service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:31:08.970796 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5cd89e-29d4-49ab-8a0a-c6ddd10dfa8b service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:31:08.971066 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5cd89e-29d4-49ab-8a0a-c6ddd10dfa8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:31:08.980419 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5cd89e-29d4-49ab-8a0a-c6ddd10dfa8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:31:08.981874 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5cd89e-29d4-49ab-8a0a-c6ddd10dfa8b service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:31:08.992105 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 761/1533] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:31:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:31:12.462587 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb77a735-1534-408d-b116-66031aea50eb None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:31:12.463146 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb77a735-1534-408d-b116-66031aea50eb None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:31:12.463146 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb77a735-1534-408d-b116-66031aea50eb None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:31:12.464490 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 773/1534] 127.0.0.1 () {54 vars in 917 bytes} [Sat Aug 6 01:31:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:31:12.469773 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b27e7d-bec5-44ed-8814-df9a2cae7e48 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:31:12.470163 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b27e7d-bec5-44ed-8814-df9a2cae7e48 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:31:12.470341 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b27e7d-bec5-44ed-8814-df9a2cae7e48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:31:12.478765 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=81 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:31:12.492632 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-d3b27e7d-bec5-44ed-8814-df9a2cae7e48 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:31:12.493857 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-d3b27e7d-bec5-44ed-8814-df9a2cae7e48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:31:12.501578 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 762/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Aug 6 01:31:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:31:12.805317 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb77a735-1534-408d-b116-66031aea50eb None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:31:12.807109 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-cb77a735-1534-408d-b116-66031aea50eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:31:12.872714 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:31:12.886256 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb77a735-1534-408d-b116-66031aea50eb None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:31:12.889936 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822fbd4d-8cde-40c0-8dc7-5379aa8863ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1be1e6173e374a60b9e1fbf51a1ab571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:31:12.890583 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822fbd4d-8cde-40c0-8dc7-5379aa8863ef service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:31:12.890861 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822fbd4d-8cde-40c0-8dc7-5379aa8863ef service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:31:12.891159 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822fbd4d-8cde-40c0-8dc7-5379aa8863ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:31:12.900578 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822fbd4d-8cde-40c0-8dc7-5379aa8863ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:31:12.902028 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822fbd4d-8cde-40c0-8dc7-5379aa8863ef service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:31:12.911210 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 774/1536] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:31:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:31:14.929442 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620c56f0-4e42-4aae-9728-fdd7898ce35f None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:31:14.929980 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620c56f0-4e42-4aae-9728-fdd7898ce35f None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:31:14.930097 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620c56f0-4e42-4aae-9728-fdd7898ce35f None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:31:14.931377 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 763/1537] 127.0.0.1 () {54 vars in 917 bytes} [Sat Aug 6 01:31:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:31:14.936615 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ebd4bc-b86a-42f2-8c13-18a1b10c1f36 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:31:14.937026 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ebd4bc-b86a-42f2-8c13-18a1b10c1f36 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:31:14.937221 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ebd4bc-b86a-42f2-8c13-18a1b10c1f36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:31:14.957146 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-11ebd4bc-b86a-42f2-8c13-18a1b10c1f36 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:31:14.958440 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-11ebd4bc-b86a-42f2-8c13-18a1b10c1f36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:31:14.966271 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 775/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Aug 6 01:31:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:31:15.268647 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620c56f0-4e42-4aae-9728-fdd7898ce35f None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:31:15.270503 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-620c56f0-4e42-4aae-9728-fdd7898ce35f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:31:15.284249 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620c56f0-4e42-4aae-9728-fdd7898ce35f None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:31:15.287978 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f39ccae-3c7f-4348-8ad7-7f27a01c8a75 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1be1e6173e374a60b9e1fbf51a1ab571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:31:15.288586 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f39ccae-3c7f-4348-8ad7-7f27a01c8a75 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:31:15.288869 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f39ccae-3c7f-4348-8ad7-7f27a01c8a75 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:31:15.289188 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f39ccae-3c7f-4348-8ad7-7f27a01c8a75 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:31:15.299130 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f39ccae-3c7f-4348-8ad7-7f27a01c8a75 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:31:15.300578 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f39ccae-3c7f-4348-8ad7-7f27a01c8a75 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:31:15.310013 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 764/1539] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:31:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:31:24.952389 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=11 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:31:25.305834 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=8 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:32:38.921152 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b51325-fa42-409c-a017-250e4967fc7c None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:38.921800 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b51325-fa42-409c-a017-250e4967fc7c None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:38.921800 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b51325-fa42-409c-a017-250e4967fc7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:38.950480 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-b6b51325-fa42-409c-a017-250e4967fc7c None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:32:38.951716 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-b6b51325-fa42-409c-a017-250e4967fc7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:32:38.961724 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 776/1540] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:32:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:32:38.969917 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a26908-e85a-4a82-92b8-0e4359969b8d None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:38.970923 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a26908-e85a-4a82-92b8-0e4359969b8d None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:38.971436 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a26908-e85a-4a82-92b8-0e4359969b8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:39.006134 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-a7a26908-e85a-4a82-92b8-0e4359969b8d None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:32:39.007766 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-a7a26908-e85a-4a82-92b8-0e4359969b8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:32:39.022427 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 765/1541] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:32:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:32:39.026471 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b70cc89-2d40-45bc-9d9a-6a4bcca22eea None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:39.026652 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a26908-e85a-4a82-92b8-0e4359969b8d None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:39.027275 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b70cc89-2d40-45bc-9d9a-6a4bcca22eea None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:39.027769 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b70cc89-2d40-45bc-9d9a-6a4bcca22eea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:39.038168 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a26908-e85a-4a82-92b8-0e4359969b8d None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:39.044730 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c207f0-76a9-4890-9aae-65075a576da7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:39.045522 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c207f0-76a9-4890-9aae-65075a576da7 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:39.045808 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c207f0-76a9-4890-9aae-65075a576da7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:39.046084 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c207f0-76a9-4890-9aae-65075a576da7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:39.047450 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c207f0-76a9-4890-9aae-65075a576da7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:32:39.047450 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c207f0-76a9-4890-9aae-65075a576da7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:39.048283 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: WARNING py.warnings [None req-a1c207f0-76a9-4890-9aae-65075a576da7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 06 01:32:39.048283 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: warnings.warn(msg) Aug 06 01:32:39.048283 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: Aug 06 01:32:39.049603 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c207f0-76a9-4890-9aae-65075a576da7 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:39.074116 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-6b70cc89-2d40-45bc-9d9a-6a4bcca22eea None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:32:39.075498 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-6b70cc89-2d40-45bc-9d9a-6a4bcca22eea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:32:39.091763 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 766/1542] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Aug 6 01:32:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:32:39.093211 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f18826-01da-4a06-9271-c951600de0ea None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:39.093837 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 777/1543] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:32:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:32:39.100733 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f18826-01da-4a06-9271-c951600de0ea None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:39.104282 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eedd153-9901-4b96-94a4-6774296ce4f5 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:39.104637 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eedd153-9901-4b96-94a4-6774296ce4f5 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:39.104805 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f18826-01da-4a06-9271-c951600de0ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:39.105029 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eedd153-9901-4b96-94a4-6774296ce4f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:39.137288 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-d5f18826-01da-4a06-9271-c951600de0ea None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:32:39.138521 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-d5f18826-01da-4a06-9271-c951600de0ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:32:39.155258 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 767/1544] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:32:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:32:39.156454 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f18826-01da-4a06-9271-c951600de0ea None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:39.157959 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-0eedd153-9901-4b96-94a4-6774296ce4f5 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:32:39.158933 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-0eedd153-9901-4b96-94a4-6774296ce4f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:32:39.172589 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f18826-01da-4a06-9271-c951600de0ea None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:39.182107 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee28b0a8-038e-4e02-b6c4-7ba6eaddae82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:39.183782 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee28b0a8-038e-4e02-b6c4-7ba6eaddae82 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:39.183782 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee28b0a8-038e-4e02-b6c4-7ba6eaddae82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:39.184183 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee28b0a8-038e-4e02-b6c4-7ba6eaddae82 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:39.186901 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee28b0a8-038e-4e02-b6c4-7ba6eaddae82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:39.187775 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 778/1545] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:32:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:32:39.189111 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f443d049-6577-4c3e-9d41-7601875a51b5 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:39.189441 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee28b0a8-038e-4e02-b6c4-7ba6eaddae82 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:39.189560 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f443d049-6577-4c3e-9d41-7601875a51b5 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:39.190006 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f443d049-6577-4c3e-9d41-7601875a51b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:39.243293 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-f443d049-6577-4c3e-9d41-7601875a51b5 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:32:39.244583 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-f443d049-6577-4c3e-9d41-7601875a51b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:32:39.261947 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 779/1546] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:32:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:32:39.263513 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45105b0b-00fe-43f6-ad45-64f38a28dc02 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:39.263801 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45105b0b-00fe-43f6-ad45-64f38a28dc02 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:39.264163 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45105b0b-00fe-43f6-ad45-64f38a28dc02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:39.269150 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 768/1547] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Aug 6 01:32:39 2022] POST /identity/v3/projects => generated 403 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:32:39.270137 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee28b0a8-038e-4e02-b6c4-7ba6eaddae82 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:39.297138 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-45105b0b-00fe-43f6-ad45-64f38a28dc02 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:32:39.298572 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-45105b0b-00fe-43f6-ad45-64f38a28dc02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:32:39.348011 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee28b0a8-038e-4e02-b6c4-7ba6eaddae82 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:39.353985 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b5d9bb-42c2-4b27-97a4-41b6292e50f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:39.354614 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b5d9bb-42c2-4b27-97a4-41b6292e50f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:39.354895 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b5d9bb-42c2-4b27-97a4-41b6292e50f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:39.355171 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b5d9bb-42c2-4b27-97a4-41b6292e50f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:39.355860 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b5d9bb-42c2-4b27-97a4-41b6292e50f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:32:39.356161 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b5d9bb-42c2-4b27-97a4-41b6292e50f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:39.357797 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b5d9bb-42c2-4b27-97a4-41b6292e50f1 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:39.370929 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 769/1548] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Aug 6 01:32:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:32:39.372222 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519a2fd6-9100-4acd-8db3-f1fdffe33a17 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:39.372523 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519a2fd6-9100-4acd-8db3-f1fdffe33a17 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:39.372824 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519a2fd6-9100-4acd-8db3-f1fdffe33a17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:39.396023 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 780/1549] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:32:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:32:39.397907 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45105b0b-00fe-43f6-ad45-64f38a28dc02 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:39.413022 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45105b0b-00fe-43f6-ad45-64f38a28dc02 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:39.413275 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-519a2fd6-9100-4acd-8db3-f1fdffe33a17 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:32:39.415009 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-519a2fd6-9100-4acd-8db3-f1fdffe33a17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:32:39.423798 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c77b30-a834-4129-83a4-4d0cc95d20e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:39.424371 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c77b30-a834-4129-83a4-4d0cc95d20e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:39.424656 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c77b30-a834-4129-83a4-4d0cc95d20e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:39.424958 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c77b30-a834-4129-83a4-4d0cc95d20e4 admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:39.430079 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c77b30-a834-4129-83a4-4d0cc95d20e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:39.430079 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c77b30-a834-4129-83a4-4d0cc95d20e4 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:39.433093 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 770/1550] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:32:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:32:39.436025 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6105c9-94c0-4c0d-ae46-08413e732f3d None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:39.436025 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6105c9-94c0-4c0d-ae46-08413e732f3d None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:39.436025 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6105c9-94c0-4c0d-ae46-08413e732f3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:39.454742 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 781/1551] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:32:39 2022] POST /identity/v3/users => generated 302 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:32:39.456278 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850f856d-a818-4fa8-b5f6-c677a375926c None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:39.456602 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850f856d-a818-4fa8-b5f6-c677a375926c None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:39.456877 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850f856d-a818-4fa8-b5f6-c677a375926c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:39.461854 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-8d6105c9-94c0-4c0d-ae46-08413e732f3d None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:32:39.463061 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-8d6105c9-94c0-4c0d-ae46-08413e732f3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:32:39.471374 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 771/1552] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:32:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:32:39.472741 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6105c9-94c0-4c0d-ae46-08413e732f3d None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:39.477256 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-850f856d-a818-4fa8-b5f6-c677a375926c None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:32:39.478590 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-850f856d-a818-4fa8-b5f6-c677a375926c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:32:39.482474 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6105c9-94c0-4c0d-ae46-08413e732f3d None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:39.486633 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 782/1553] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:32:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:32:39.487046 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c30b25b-3dd4-49cd-9316-c3cc7b3d1dab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:39.487649 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c30b25b-3dd4-49cd-9316-c3cc7b3d1dab admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:39.488500 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3f3b2a-6b59-485c-87f4-fc34ecf96399 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:39.488684 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c30b25b-3dd4-49cd-9316-c3cc7b3d1dab admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:39.488787 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3f3b2a-6b59-485c-87f4-fc34ecf96399 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:39.488982 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c30b25b-3dd4-49cd-9316-c3cc7b3d1dab admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:39.489331 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3f3b2a-6b59-485c-87f4-fc34ecf96399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:39.489663 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c30b25b-3dd4-49cd-9316-c3cc7b3d1dab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:39.491568 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c30b25b-3dd4-49cd-9316-c3cc7b3d1dab admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:39.508606 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-cc3f3b2a-6b59-485c-87f4-fc34ecf96399 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:32:39.510044 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-cc3f3b2a-6b59-485c-87f4-fc34ecf96399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:32:39.518397 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 783/1554] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:32:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:32:39.519921 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3f3b2a-6b59-485c-87f4-fc34ecf96399 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:39.523477 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 772/1555] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Aug 6 01:32:39 2022] POST /identity/v3/projects => generated 377 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:32:39.523477 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2907804b-46e3-4bc5-8019-66cfa57c2586 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:39.523477 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2907804b-46e3-4bc5-8019-66cfa57c2586 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:39.523477 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2907804b-46e3-4bc5-8019-66cfa57c2586 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:39.532674 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3f3b2a-6b59-485c-87f4-fc34ecf96399 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:39.543476 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-2907804b-46e3-4bc5-8019-66cfa57c2586 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:32:39.544684 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-2907804b-46e3-4bc5-8019-66cfa57c2586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:32:39.589520 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bf52ed-c872-4a49-b5a4-eadc4aab0be4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:39.590146 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bf52ed-c872-4a49-b5a4-eadc4aab0be4 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:39.590424 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bf52ed-c872-4a49-b5a4-eadc4aab0be4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:39.590709 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bf52ed-c872-4a49-b5a4-eadc4aab0be4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:39.591512 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bf52ed-c872-4a49-b5a4-eadc4aab0be4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:32:39.591827 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bf52ed-c872-4a49-b5a4-eadc4aab0be4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:39.593476 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bf52ed-c872-4a49-b5a4-eadc4aab0be4 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:39.604837 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 773/1556] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:32:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:32:39.606139 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2400db82-e188-44df-aee2-f9c7aa2b0c31 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:39.606496 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2400db82-e188-44df-aee2-f9c7aa2b0c31 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:39.606777 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2400db82-e188-44df-aee2-f9c7aa2b0c31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:39.609207 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 784/1557] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Aug 6 01:32:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:32:39.610577 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bf52ed-c872-4a49-b5a4-eadc4aab0be4 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:39.615354 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:32:39.622541 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bf52ed-c872-4a49-b5a4-eadc4aab0be4 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:39.627676 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8780603-7a45-41ae-928d-c9c12b2fa64a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:39.628353 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8780603-7a45-41ae-928d-c9c12b2fa64a admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:39.628641 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8780603-7a45-41ae-928d-c9c12b2fa64a admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:39.628921 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8780603-7a45-41ae-928d-c9c12b2fa64a admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:39.629127 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-2400db82-e188-44df-aee2-f9c7aa2b0c31 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:32:39.629673 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8780603-7a45-41ae-928d-c9c12b2fa64a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:39.630238 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-2400db82-e188-44df-aee2-f9c7aa2b0c31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:32:39.631271 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8780603-7a45-41ae-928d-c9c12b2fa64a admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:39.638290 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 774/1558] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:32:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:32:39.639066 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2400db82-e188-44df-aee2-f9c7aa2b0c31 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:39.646418 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 785/1559] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:32:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:32:39.646418 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757bf229-2c06-4069-8729-e63d2b6be688 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:39.646418 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757bf229-2c06-4069-8729-e63d2b6be688 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:39.646418 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757bf229-2c06-4069-8729-e63d2b6be688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:39.652045 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2400db82-e188-44df-aee2-f9c7aa2b0c31 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:39.656815 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1fc016-83c0-4cd3-8667-778ffc8b100b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:39.657451 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1fc016-83c0-4cd3-8667-778ffc8b100b admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:39.657731 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1fc016-83c0-4cd3-8667-778ffc8b100b admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:39.658006 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1fc016-83c0-4cd3-8667-778ffc8b100b admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:39.658699 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1fc016-83c0-4cd3-8667-778ffc8b100b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:32:39.659003 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1fc016-83c0-4cd3-8667-778ffc8b100b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:39.660947 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1fc016-83c0-4cd3-8667-778ffc8b100b admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:39.664255 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-757bf229-2c06-4069-8729-e63d2b6be688 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:32:39.665493 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-757bf229-2c06-4069-8729-e63d2b6be688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:32:39.672716 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 775/1560] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Aug 6 01:32:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:32:39.673506 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1fc016-83c0-4cd3-8667-778ffc8b100b admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:39.677836 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 786/1561] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:32:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:32:39.679106 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c0209f-23df-420a-aaaf-925b6463d8c1 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:39.679493 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c0209f-23df-420a-aaaf-925b6463d8c1 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:39.680059 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c0209f-23df-420a-aaaf-925b6463d8c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:39.685608 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1fc016-83c0-4cd3-8667-778ffc8b100b admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:39.690300 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab58adcc-0f9f-422f-b13a-62a853e591f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:39.691471 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab58adcc-0f9f-422f-b13a-62a853e591f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:39.691763 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab58adcc-0f9f-422f-b13a-62a853e591f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:39.692069 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab58adcc-0f9f-422f-b13a-62a853e591f0 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:39.692730 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab58adcc-0f9f-422f-b13a-62a853e591f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:39.693773 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: WARNING py.warnings [None req-ab58adcc-0f9f-422f-b13a-62a853e591f0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 06 01:32:39.693773 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: warnings.warn(msg) Aug 06 01:32:39.693773 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: Aug 06 01:32:39.694984 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab58adcc-0f9f-422f-b13a-62a853e591f0 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:39.699645 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-70c0209f-23df-420a-aaaf-925b6463d8c1 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:32:39.700818 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-70c0209f-23df-420a-aaaf-925b6463d8c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:32:39.710227 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 787/1562] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:32:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:32:39.711430 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e88eb1-135f-47e9-9a22-3821c8e495a6 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:39.711715 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e88eb1-135f-47e9-9a22-3821c8e495a6 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:39.711981 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e88eb1-135f-47e9-9a22-3821c8e495a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:39.720923 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 776/1563] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:32:39 2022] POST /identity/v3/users => generated 289 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:32:39.720923 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab58adcc-0f9f-422f-b13a-62a853e591f0 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:39.728630 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab58adcc-0f9f-422f-b13a-62a853e591f0 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:39.733194 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-63e88eb1-135f-47e9-9a22-3821c8e495a6 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:32:39.733358 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591479be-45ff-4882-819a-052a50164e77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:39.733931 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591479be-45ff-4882-819a-052a50164e77 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:39.734216 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591479be-45ff-4882-819a-052a50164e77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:39.734496 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591479be-45ff-4882-819a-052a50164e77 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:39.734907 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-63e88eb1-135f-47e9-9a22-3821c8e495a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:32:39.735160 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591479be-45ff-4882-819a-052a50164e77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:39.737088 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591479be-45ff-4882-819a-052a50164e77 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:39.744564 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 788/1564] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:32:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:32:39.745488 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e88eb1-135f-47e9-9a22-3821c8e495a6 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:39.756167 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e88eb1-135f-47e9-9a22-3821c8e495a6 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:39.761711 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda9234b-0f96-4871-b613-d93187e3cf71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:39.762436 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda9234b-0f96-4871-b613-d93187e3cf71 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:39.762718 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda9234b-0f96-4871-b613-d93187e3cf71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:39.762993 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda9234b-0f96-4871-b613-d93187e3cf71 admin admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:39.763679 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda9234b-0f96-4871-b613-d93187e3cf71 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:32:39.763983 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda9234b-0f96-4871-b613-d93187e3cf71 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:39.765785 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda9234b-0f96-4871-b613-d93187e3cf71 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:39.769641 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 777/1565] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Aug 6 01:32:39 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:32:39.769789 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:32:39.769922 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591479be-45ff-4882-819a-052a50164e77 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:39.780657 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 789/1566] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Aug 6 01:32:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:32:39.781730 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda9234b-0f96-4871-b613-d93187e3cf71 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:39.791691 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda9234b-0f96-4871-b613-d93187e3cf71 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:39.836032 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591479be-45ff-4882-819a-052a50164e77 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:39.840761 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dea378e-0790-497c-a9b3-6360dce1b34c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:39.841421 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dea378e-0790-497c-a9b3-6360dce1b34c admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:39.841701 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dea378e-0790-497c-a9b3-6360dce1b34c admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:39.841972 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dea378e-0790-497c-a9b3-6360dce1b34c admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:39.842655 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dea378e-0790-497c-a9b3-6360dce1b34c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:39.844241 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dea378e-0790-497c-a9b3-6360dce1b34c admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:39.849224 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fed066-09dd-48d8-8d58-0e38ee043994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:39.849850 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fed066-09dd-48d8-8d58-0e38ee043994 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:39.850138 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fed066-09dd-48d8-8d58-0e38ee043994 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:39.850435 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fed066-09dd-48d8-8d58-0e38ee043994 admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:39.851104 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fed066-09dd-48d8-8d58-0e38ee043994 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:39.852119 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: WARNING py.warnings [None req-64fed066-09dd-48d8-8d58-0e38ee043994 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 06 01:32:39.852119 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: warnings.warn(msg) Aug 06 01:32:39.852119 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: Aug 06 01:32:39.853346 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fed066-09dd-48d8-8d58-0e38ee043994 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:39.854996 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 778/1567] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:32:39 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:32:39.855851 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dea378e-0790-497c-a9b3-6360dce1b34c admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:39.867937 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dea378e-0790-497c-a9b3-6360dce1b34c admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:39.873211 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd0dda3-cfe1-4a2a-a1b8-1a5c09167509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:39.873841 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd0dda3-cfe1-4a2a-a1b8-1a5c09167509 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:39.874121 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd0dda3-cfe1-4a2a-a1b8-1a5c09167509 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:39.874392 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd0dda3-cfe1-4a2a-a1b8-1a5c09167509 admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:39.875141 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd0dda3-cfe1-4a2a-a1b8-1a5c09167509 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:32:39.875421 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd0dda3-cfe1-4a2a-a1b8-1a5c09167509 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:39.877005 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd0dda3-cfe1-4a2a-a1b8-1a5c09167509 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:39.885519 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 790/1568] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Aug 6 01:32:39 2022] POST /identity/v3/projects => generated 367 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:32:39.885519 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fed066-09dd-48d8-8d58-0e38ee043994 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:39.897833 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 779/1569] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Aug 6 01:32:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:32:39.897833 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd0dda3-cfe1-4a2a-a1b8-1a5c09167509 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:39.902996 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd0dda3-cfe1-4a2a-a1b8-1a5c09167509 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:39.952250 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fed066-09dd-48d8-8d58-0e38ee043994 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:39.957070 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffe9af3-de4f-4a59-a07b-1ad583ecec64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:39.957698 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffe9af3-de4f-4a59-a07b-1ad583ecec64 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:39.957983 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffe9af3-de4f-4a59-a07b-1ad583ecec64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:39.958269 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffe9af3-de4f-4a59-a07b-1ad583ecec64 admin admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:39.958969 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffe9af3-de4f-4a59-a07b-1ad583ecec64 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:32:39.959124 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93e6269-976f-41be-abba-c13776a7764e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:39.959281 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffe9af3-de4f-4a59-a07b-1ad583ecec64 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:39.959736 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93e6269-976f-41be-abba-c13776a7764e admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:39.960025 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93e6269-976f-41be-abba-c13776a7764e admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:39.960299 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93e6269-976f-41be-abba-c13776a7764e admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:39.960895 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffe9af3-de4f-4a59-a07b-1ad583ecec64 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:39.961020 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93e6269-976f-41be-abba-c13776a7764e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:39.962566 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93e6269-976f-41be-abba-c13776a7764e admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:39.972835 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 791/1570] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Aug 6 01:32:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:32:39.977340 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 780/1571] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:32:39 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:32:39.977340 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93e6269-976f-41be-abba-c13776a7764e admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:39.978820 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffe9af3-de4f-4a59-a07b-1ad583ecec64 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:39.985017 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93e6269-976f-41be-abba-c13776a7764e admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:39.989573 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffe9af3-de4f-4a59-a07b-1ad583ecec64 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:39.990473 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83771e47-d732-4032-8b9a-bdb486fa076d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:39.991077 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83771e47-d732-4032-8b9a-bdb486fa076d admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:39.991351 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83771e47-d732-4032-8b9a-bdb486fa076d admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:39.991628 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83771e47-d732-4032-8b9a-bdb486fa076d admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:39.992281 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83771e47-d732-4032-8b9a-bdb486fa076d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:39.994215 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83771e47-d732-4032-8b9a-bdb486fa076d admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:39.994495 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0782c8-3415-4e23-b59c-843db2673d20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:39.995109 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0782c8-3415-4e23-b59c-843db2673d20 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:39.995396 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0782c8-3415-4e23-b59c-843db2673d20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:39.995675 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0782c8-3415-4e23-b59c-843db2673d20 admin admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:39.996366 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0782c8-3415-4e23-b59c-843db2673d20 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:32:39.996680 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0782c8-3415-4e23-b59c-843db2673d20 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:39.998327 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0782c8-3415-4e23-b59c-843db2673d20 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:40.011035 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 792/1572] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Aug 6 01:32:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:32:40.011776 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0782c8-3415-4e23-b59c-843db2673d20 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:40.018645 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 781/1573] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:32:39 2022] POST /identity/v3/users => generated 287 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:32:40.019611 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83771e47-d732-4032-8b9a-bdb486fa076d admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:40.023071 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0782c8-3415-4e23-b59c-843db2673d20 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:40.026129 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:32:40.028099 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cb5a32-66b6-45eb-9395-8e64a56a85ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:40.028769 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cb5a32-66b6-45eb-9395-8e64a56a85ee admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:40.029082 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cb5a32-66b6-45eb-9395-8e64a56a85ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:40.029353 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cb5a32-66b6-45eb-9395-8e64a56a85ee admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:40.030014 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cb5a32-66b6-45eb-9395-8e64a56a85ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:40.030281 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83771e47-d732-4032-8b9a-bdb486fa076d admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:40.031953 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cb5a32-66b6-45eb-9395-8e64a56a85ee admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:40.037303 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62f7117-5824-4af0-8d5a-d55f8b7d1be1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:40.037970 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62f7117-5824-4af0-8d5a-d55f8b7d1be1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:40.038558 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62f7117-5824-4af0-8d5a-d55f8b7d1be1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:40.038558 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62f7117-5824-4af0-8d5a-d55f8b7d1be1 admin admin] PATH_INFO: `/v3/projects/08eb0e0a70fb4b7baf477606bc46f603/users/bfb318d8b4d7478a92bcfac9a07a447f/roles/61b3b93d37a047e4bb8369b46fe20db9` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:40.054645 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62f7117-5824-4af0-8d5a-d55f8b7d1be1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08eb0e0a70fb4b7baf477606bc46f603, user_id=bfb318d8b4d7478a92bcfac9a07a447f, role_id=61b3b93d37a047e4bb8369b46fe20db9)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:40.057363 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62f7117-5824-4af0-8d5a-d55f8b7d1be1 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:40.060156 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 793/1574] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Aug 6 01:32:40 2022] POST /identity/v3/projects => generated 389 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:32:40.060962 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cb5a32-66b6-45eb-9395-8e64a56a85ee admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:40.075359 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 782/1575] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:32:40 2022] PUT /identity/v3/projects/08eb0e0a70fb4b7baf477606bc46f603/users/bfb318d8b4d7478a92bcfac9a07a447f/roles/61b3b93d37a047e4bb8369b46fe20db9 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:32:40.076154 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62f7117-5824-4af0-8d5a-d55f8b7d1be1 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:40.104053 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:32:40.127263 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cb5a32-66b6-45eb-9395-8e64a56a85ee admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:40.148005 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62f7117-5824-4af0-8d5a-d55f8b7d1be1 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:40.152675 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93076587-4774-4383-9af4-550c676ddcf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:40.153333 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93076587-4774-4383-9af4-550c676ddcf9 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:40.153621 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93076587-4774-4383-9af4-550c676ddcf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:40.153902 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93076587-4774-4383-9af4-550c676ddcf9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:40.154581 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93076587-4774-4383-9af4-550c676ddcf9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:40.156526 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93076587-4774-4383-9af4-550c676ddcf9 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:40.186644 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 783/1576] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Aug 6 01:32:40 2022] POST /identity/v3/projects => generated 363 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:32:40.186644 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93076587-4774-4383-9af4-550c676ddcf9 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:40.188796 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d246e639-af4f-4061-93f5-16f5523d8a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:40.189459 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d246e639-af4f-4061-93f5-16f5523d8a1a admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:40.189742 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d246e639-af4f-4061-93f5-16f5523d8a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:40.190017 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d246e639-af4f-4061-93f5-16f5523d8a1a admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:40.190678 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d246e639-af4f-4061-93f5-16f5523d8a1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:40.193946 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d246e639-af4f-4061-93f5-16f5523d8a1a admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:40.216198 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 794/1577] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:32:40 2022] POST /identity/v3/users => generated 284 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:32:40.217143 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d246e639-af4f-4061-93f5-16f5523d8a1a admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:40.227416 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d246e639-af4f-4061-93f5-16f5523d8a1a admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:40.250689 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93076587-4774-4383-9af4-550c676ddcf9 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:40.256256 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2904e7d4-e6e7-46ed-824c-7816fe172ace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:40.256870 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2904e7d4-e6e7-46ed-824c-7816fe172ace admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:40.257182 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2904e7d4-e6e7-46ed-824c-7816fe172ace admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:40.257456 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2904e7d4-e6e7-46ed-824c-7816fe172ace admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:40.258145 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2904e7d4-e6e7-46ed-824c-7816fe172ace admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:40.259771 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2904e7d4-e6e7-46ed-824c-7816fe172ace admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:40.269571 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 784/1578] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:32:40 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:32:40.270346 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2904e7d4-e6e7-46ed-824c-7816fe172ace admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:40.279460 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2904e7d4-e6e7-46ed-824c-7816fe172ace admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:40.284013 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a00208-6f66-40f2-8f92-1029ebed0231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:40.284553 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3de0534-993b-4dc0-bce7-47351cabf85a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:40.284705 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a00208-6f66-40f2-8f92-1029ebed0231 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:40.285074 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a00208-6f66-40f2-8f92-1029ebed0231 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:40.285352 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a00208-6f66-40f2-8f92-1029ebed0231 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:40.285535 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3de0534-993b-4dc0-bce7-47351cabf85a admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:40.285856 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3de0534-993b-4dc0-bce7-47351cabf85a admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:40.286134 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3de0534-993b-4dc0-bce7-47351cabf85a admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:40.286226 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a00208-6f66-40f2-8f92-1029ebed0231 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:40.286794 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3de0534-993b-4dc0-bce7-47351cabf85a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:40.288069 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a00208-6f66-40f2-8f92-1029ebed0231 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:40.288820 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3de0534-993b-4dc0-bce7-47351cabf85a admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:40.317225 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 795/1579] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Aug 6 01:32:40 2022] POST /identity/v3/projects => generated 393 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:32:40.320672 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 785/1580] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Aug 6 01:32:40 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:32:40.321767 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a00208-6f66-40f2-8f92-1029ebed0231 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:40.321883 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3de0534-993b-4dc0-bce7-47351cabf85a admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:40.365391 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:32:40.387753 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a00208-6f66-40f2-8f92-1029ebed0231 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:40.388398 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3de0534-993b-4dc0-bce7-47351cabf85a admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:40.393176 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c12026-fd11-4ec3-9286-f09b90e11260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:40.393346 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20acc671-a55b-4c15-98f1-1f7392d40578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:40.393808 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c12026-fd11-4ec3-9286-f09b90e11260 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:40.393960 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20acc671-a55b-4c15-98f1-1f7392d40578 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:40.394767 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20acc671-a55b-4c15-98f1-1f7392d40578 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:40.394872 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c12026-fd11-4ec3-9286-f09b90e11260 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:40.395017 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c12026-fd11-4ec3-9286-f09b90e11260 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:40.395114 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20acc671-a55b-4c15-98f1-1f7392d40578 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:40.395692 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c12026-fd11-4ec3-9286-f09b90e11260 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:40.395803 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20acc671-a55b-4c15-98f1-1f7392d40578 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:40.397323 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c12026-fd11-4ec3-9286-f09b90e11260 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:40.397725 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20acc671-a55b-4c15-98f1-1f7392d40578 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:40.398865 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:32:40.413466 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 796/1581] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:32:40 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:32:40.414914 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f517f4c2-4bc3-4fa5-988c-53e153b072d9 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:40.415522 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f517f4c2-4bc3-4fa5-988c-53e153b072d9 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:40.416019 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f517f4c2-4bc3-4fa5-988c-53e153b072d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:40.420769 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 786/1582] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:32:40 2022] POST /identity/v3/users => generated 295 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:32:40.421487 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20acc671-a55b-4c15-98f1-1f7392d40578 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:40.431528 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20acc671-a55b-4c15-98f1-1f7392d40578 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:40.436281 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45a9f4e-432e-4433-b33e-fdbb15b5690d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:40.436878 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45a9f4e-432e-4433-b33e-fdbb15b5690d admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:40.437190 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45a9f4e-432e-4433-b33e-fdbb15b5690d admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:40.437473 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45a9f4e-432e-4433-b33e-fdbb15b5690d admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:40.438127 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45a9f4e-432e-4433-b33e-fdbb15b5690d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:40.440035 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45a9f4e-432e-4433-b33e-fdbb15b5690d admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:40.444167 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-f517f4c2-4bc3-4fa5-988c-53e153b072d9 None None] MFA Rules not processed for user `bfb318d8b4d7478a92bcfac9a07a447f`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:32:40.445420 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-f517f4c2-4bc3-4fa5-988c-53e153b072d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:32:40.461546 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 787/1583] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:32:40 2022] POST /identity/v3/users => generated 282 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:32:40.462357 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45a9f4e-432e-4433-b33e-fdbb15b5690d admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:40.472838 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45a9f4e-432e-4433-b33e-fdbb15b5690d admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:40.478079 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5b4387-f57a-460c-a3ac-ca5f08ac8081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:40.478681 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5b4387-f57a-460c-a3ac-ca5f08ac8081 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:40.478967 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5b4387-f57a-460c-a3ac-ca5f08ac8081 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:40.479246 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5b4387-f57a-460c-a3ac-ca5f08ac8081 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:40.479931 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5b4387-f57a-460c-a3ac-ca5f08ac8081 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:40.481554 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5b4387-f57a-460c-a3ac-ca5f08ac8081 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:40.492670 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 788/1584] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:32:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:32:40.493463 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5b4387-f57a-460c-a3ac-ca5f08ac8081 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:40.502760 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5b4387-f57a-460c-a3ac-ca5f08ac8081 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:40.507662 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942e05d0-2e60-4154-802e-dd81ceb4a90a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:40.508283 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942e05d0-2e60-4154-802e-dd81ceb4a90a admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:40.508561 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942e05d0-2e60-4154-802e-dd81ceb4a90a admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:40.508834 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942e05d0-2e60-4154-802e-dd81ceb4a90a admin admin] PATH_INFO: `/v3/projects/b124d9abc46d4e85927cf1b40dd4a171/users/5823f66f313843ddac68ff7a951321ae/roles/61b3b93d37a047e4bb8369b46fe20db9` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:40.517460 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 797/1585] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:32:40 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:32:40.521650 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f517f4c2-4bc3-4fa5-988c-53e153b072d9 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:40.523840 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942e05d0-2e60-4154-802e-dd81ceb4a90a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b124d9abc46d4e85927cf1b40dd4a171, user_id=5823f66f313843ddac68ff7a951321ae, role_id=61b3b93d37a047e4bb8369b46fe20db9)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:40.526530 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942e05d0-2e60-4154-802e-dd81ceb4a90a admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:40.529271 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f517f4c2-4bc3-4fa5-988c-53e153b072d9 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:40.534216 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6e13a1-3681-46f3-a8f9-671c2eccab7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:40.535423 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6e13a1-3681-46f3-a8f9-671c2eccab7e admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:40.535713 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6e13a1-3681-46f3-a8f9-671c2eccab7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:40.535970 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6e13a1-3681-46f3-a8f9-671c2eccab7e admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:40.536641 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6e13a1-3681-46f3-a8f9-671c2eccab7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:40.538598 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6e13a1-3681-46f3-a8f9-671c2eccab7e admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:40.540643 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 789/1586] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:32:40 2022] PUT /identity/v3/projects/b124d9abc46d4e85927cf1b40dd4a171/users/5823f66f313843ddac68ff7a951321ae/roles/61b3b93d37a047e4bb8369b46fe20db9 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:32:40.541453 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942e05d0-2e60-4154-802e-dd81ceb4a90a admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:40.559318 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 798/1587] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:32:40 2022] POST /identity/v3/users => generated 297 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:32:40.560397 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6e13a1-3681-46f3-a8f9-671c2eccab7e admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:40.570298 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6e13a1-3681-46f3-a8f9-671c2eccab7e admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:40.605247 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942e05d0-2e60-4154-802e-dd81ceb4a90a admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:40.610159 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116b7701-7985-410e-98d5-e24ff8c1e96d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:40.610784 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116b7701-7985-410e-98d5-e24ff8c1e96d admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:40.611100 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116b7701-7985-410e-98d5-e24ff8c1e96d admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:40.611380 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116b7701-7985-410e-98d5-e24ff8c1e96d admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:40.612048 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116b7701-7985-410e-98d5-e24ff8c1e96d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:40.614291 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116b7701-7985-410e-98d5-e24ff8c1e96d admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:40.626952 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a48dedf-9bdb-49f0-8a3d-1316336cbe34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:40.627638 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a48dedf-9bdb-49f0-8a3d-1316336cbe34 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:40.627916 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a48dedf-9bdb-49f0-8a3d-1316336cbe34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:40.628252 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a48dedf-9bdb-49f0-8a3d-1316336cbe34 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:40.629741 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a48dedf-9bdb-49f0-8a3d-1316336cbe34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:40.631395 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a48dedf-9bdb-49f0-8a3d-1316336cbe34 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:40.639994 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 790/1588] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:32:40 2022] POST /identity/v3/users => generated 288 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:32:40.639994 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116b7701-7985-410e-98d5-e24ff8c1e96d admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:40.643654 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 799/1589] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:32:40 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:32:40.645219 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a48dedf-9bdb-49f0-8a3d-1316336cbe34 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:40.648307 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116b7701-7985-410e-98d5-e24ff8c1e96d admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:40.653011 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c491974-2c8b-4c32-9781-4ca3f665616f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:40.653647 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c491974-2c8b-4c32-9781-4ca3f665616f admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:40.653932 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c491974-2c8b-4c32-9781-4ca3f665616f admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:40.654209 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c491974-2c8b-4c32-9781-4ca3f665616f admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:40.654891 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c491974-2c8b-4c32-9781-4ca3f665616f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:40.656476 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c491974-2c8b-4c32-9781-4ca3f665616f admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:40.656851 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a48dedf-9bdb-49f0-8a3d-1316336cbe34 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:40.662637 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e974a839-78f6-4f40-abcb-efe5d5d1d97a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:40.663243 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e974a839-78f6-4f40-abcb-efe5d5d1d97a admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:40.663531 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e974a839-78f6-4f40-abcb-efe5d5d1d97a admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:40.663806 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e974a839-78f6-4f40-abcb-efe5d5d1d97a admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:40.664498 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e974a839-78f6-4f40-abcb-efe5d5d1d97a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:40.666111 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e974a839-78f6-4f40-abcb-efe5d5d1d97a admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:40.666622 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 791/1590] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:32:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:32:40.667406 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c491974-2c8b-4c32-9781-4ca3f665616f admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:40.677466 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c491974-2c8b-4c32-9781-4ca3f665616f admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:40.679275 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 800/1591] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:32:40 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:32:40.680071 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e974a839-78f6-4f40-abcb-efe5d5d1d97a admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:40.682206 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9bda1f-77f5-401e-b703-b4d332f971bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:40.682818 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9bda1f-77f5-401e-b703-b4d332f971bb admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:40.683099 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9bda1f-77f5-401e-b703-b4d332f971bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:40.683377 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9bda1f-77f5-401e-b703-b4d332f971bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:40.684058 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9bda1f-77f5-401e-b703-b4d332f971bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:40.685672 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9bda1f-77f5-401e-b703-b4d332f971bb admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:40.689637 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e974a839-78f6-4f40-abcb-efe5d5d1d97a admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:40.694903 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b717690-fda9-4635-bf26-f007ea204edb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:40.695495 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b717690-fda9-4635-bf26-f007ea204edb admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:40.695779 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b717690-fda9-4635-bf26-f007ea204edb admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:40.696054 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b717690-fda9-4635-bf26-f007ea204edb admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:40.697016 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b717690-fda9-4635-bf26-f007ea204edb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:40.697170 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 792/1592] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:32:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:32:40.698101 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77db5d12-a077-4633-93d3-b60d25849e69 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:40.698547 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77db5d12-a077-4633-93d3-b60d25849e69 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:40.698773 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b717690-fda9-4635-bf26-f007ea204edb admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:40.698879 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77db5d12-a077-4633-93d3-b60d25849e69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:40.717093 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-77db5d12-a077-4633-93d3-b60d25849e69 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:32:40.718336 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-77db5d12-a077-4633-93d3-b60d25849e69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:32:40.729161 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 801/1593] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Aug 6 01:32:40 2022] POST /identity/v3/projects => generated 403 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:32:40.729161 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b717690-fda9-4635-bf26-f007ea204edb admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:40.729385 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 793/1594] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:32:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:32:40.729385 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77db5d12-a077-4633-93d3-b60d25849e69 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:40.740183 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77db5d12-a077-4633-93d3-b60d25849e69 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:40.766334 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:32:40.773319 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:32:40.798092 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273cfae2-13fb-4497-86e4-7a572f3c409e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:40.798280 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b717690-fda9-4635-bf26-f007ea204edb admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:40.798717 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273cfae2-13fb-4497-86e4-7a572f3c409e admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:40.799007 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273cfae2-13fb-4497-86e4-7a572f3c409e admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:40.799288 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273cfae2-13fb-4497-86e4-7a572f3c409e admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:40.800012 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273cfae2-13fb-4497-86e4-7a572f3c409e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:40.802363 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273cfae2-13fb-4497-86e4-7a572f3c409e admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:40.803438 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac641a0-7790-4565-8e12-f2af3978faf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:40.804053 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac641a0-7790-4565-8e12-f2af3978faf1 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:40.804334 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac641a0-7790-4565-8e12-f2af3978faf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:40.804612 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac641a0-7790-4565-8e12-f2af3978faf1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:40.805342 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac641a0-7790-4565-8e12-f2af3978faf1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:40.806933 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac641a0-7790-4565-8e12-f2af3978faf1 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:40.813004 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 794/1595] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:32:40 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:32:40.814008 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273cfae2-13fb-4497-86e4-7a572f3c409e admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:40.817751 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 802/1596] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:32:40 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:32:40.818450 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac641a0-7790-4565-8e12-f2af3978faf1 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:40.825363 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273cfae2-13fb-4497-86e4-7a572f3c409e admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:40.827805 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac641a0-7790-4565-8e12-f2af3978faf1 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:40.830048 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1adaa7-8244-47b2-8220-35d66ac7fca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:40.830707 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1adaa7-8244-47b2-8220-35d66ac7fca4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:40.830991 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1adaa7-8244-47b2-8220-35d66ac7fca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:40.831274 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1adaa7-8244-47b2-8220-35d66ac7fca4 admin admin] PATH_INFO: `/v3/projects/a4a60a35d1b246babe9832dc26bb4450/users/f9fa23907107401ebf143f7e550ae149/roles/61b3b93d37a047e4bb8369b46fe20db9` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:40.832966 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9c8995-0269-408f-b4d1-2b80837ac8ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:40.833588 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9c8995-0269-408f-b4d1-2b80837ac8ec admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:40.833871 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9c8995-0269-408f-b4d1-2b80837ac8ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:40.834152 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9c8995-0269-408f-b4d1-2b80837ac8ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:40.834834 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9c8995-0269-408f-b4d1-2b80837ac8ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:40.836406 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9c8995-0269-408f-b4d1-2b80837ac8ec admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:40.846138 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 803/1597] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:32:40 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:32:40.846918 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9c8995-0269-408f-b4d1-2b80837ac8ec admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:40.847245 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1adaa7-8244-47b2-8220-35d66ac7fca4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4a60a35d1b246babe9832dc26bb4450, user_id=f9fa23907107401ebf143f7e550ae149, role_id=61b3b93d37a047e4bb8369b46fe20db9)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:40.849953 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1adaa7-8244-47b2-8220-35d66ac7fca4 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:40.857462 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9c8995-0269-408f-b4d1-2b80837ac8ec admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:40.863497 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 795/1598] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:32:40 2022] PUT /identity/v3/projects/a4a60a35d1b246babe9832dc26bb4450/users/f9fa23907107401ebf143f7e550ae149/roles/61b3b93d37a047e4bb8369b46fe20db9 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:32:40.864328 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1adaa7-8244-47b2-8220-35d66ac7fca4 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:40.916842 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9e10c3-e585-4f63-ab3a-f348609a2516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:40.917511 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9e10c3-e585-4f63-ab3a-f348609a2516 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:40.917803 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9e10c3-e585-4f63-ab3a-f348609a2516 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:40.918085 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9e10c3-e585-4f63-ab3a-f348609a2516 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:40.918782 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9e10c3-e585-4f63-ab3a-f348609a2516 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:40.920385 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9e10c3-e585-4f63-ab3a-f348609a2516 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:40.930672 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 804/1599] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:32:40 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:32:40.931459 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9e10c3-e585-4f63-ab3a-f348609a2516 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:40.934488 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1adaa7-8244-47b2-8220-35d66ac7fca4 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:40.939361 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740571d9-9431-45c9-921c-470c5071f98f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:40.940029 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740571d9-9431-45c9-921c-470c5071f98f admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:40.940313 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740571d9-9431-45c9-921c-470c5071f98f admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:40.940596 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740571d9-9431-45c9-921c-470c5071f98f admin admin] PATH_INFO: `/v3/projects/a778a936bf3c45d39e51c562209dfb64/users/d588a87dd3bc4a97b75f06491445a7a8/roles/61b3b93d37a047e4bb8369b46fe20db9` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:40.941358 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9e10c3-e585-4f63-ab3a-f348609a2516 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:40.946207 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590085d6-f392-45d2-9cd1-fd7c135d5b10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:40.946803 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590085d6-f392-45d2-9cd1-fd7c135d5b10 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:40.947086 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590085d6-f392-45d2-9cd1-fd7c135d5b10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:40.947361 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590085d6-f392-45d2-9cd1-fd7c135d5b10 admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:40.948018 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590085d6-f392-45d2-9cd1-fd7c135d5b10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:40.949988 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590085d6-f392-45d2-9cd1-fd7c135d5b10 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:40.956395 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740571d9-9431-45c9-921c-470c5071f98f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a778a936bf3c45d39e51c562209dfb64, user_id=d588a87dd3bc4a97b75f06491445a7a8, role_id=61b3b93d37a047e4bb8369b46fe20db9)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:40.959621 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740571d9-9431-45c9-921c-470c5071f98f admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:40.975151 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 805/1600] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:32:40 2022] POST /identity/v3/users => generated 308 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:32:40.975677 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 796/1601] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:32:40 2022] PUT /identity/v3/projects/a778a936bf3c45d39e51c562209dfb64/users/d588a87dd3bc4a97b75f06491445a7a8/roles/61b3b93d37a047e4bb8369b46fe20db9 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:32:40.975794 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590085d6-f392-45d2-9cd1-fd7c135d5b10 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:40.975926 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740571d9-9431-45c9-921c-470c5071f98f admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:41.033679 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590085d6-f392-45d2-9cd1-fd7c135d5b10 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:41.038516 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740571d9-9431-45c9-921c-470c5071f98f admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:41.043253 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695f0764-e758-4182-9376-d766f3744455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:41.043883 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695f0764-e758-4182-9376-d766f3744455 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:41.044169 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695f0764-e758-4182-9376-d766f3744455 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:41.044450 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695f0764-e758-4182-9376-d766f3744455 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:41.045188 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695f0764-e758-4182-9376-d766f3744455 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:41.046783 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695f0764-e758-4182-9376-d766f3744455 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:41.057233 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 797/1602] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:32:40 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:32:41.058190 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695f0764-e758-4182-9376-d766f3744455 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:41.068339 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695f0764-e758-4182-9376-d766f3744455 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:41.074092 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ca7153-48b1-4d80-81a9-f47c48372006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:41.074699 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ca7153-48b1-4d80-81a9-f47c48372006 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:41.074983 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ca7153-48b1-4d80-81a9-f47c48372006 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:41.075259 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ca7153-48b1-4d80-81a9-f47c48372006 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:41.075943 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ca7153-48b1-4d80-81a9-f47c48372006 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:41.077631 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ca7153-48b1-4d80-81a9-f47c48372006 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:41.089080 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 798/1603] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:32:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:32:41.090935 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0000b84a-5ede-4759-abe1-2fbc36b75bfe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:41.091562 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0000b84a-5ede-4759-abe1-2fbc36b75bfe service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:41.091860 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0000b84a-5ede-4759-abe1-2fbc36b75bfe service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:41.092155 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0000b84a-5ede-4759-abe1-2fbc36b75bfe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:41.092768 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ca7153-48b1-4d80-81a9-f47c48372006 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:41.098598 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:32:41.102313 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0000b84a-5ede-4759-abe1-2fbc36b75bfe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:41.102859 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ca7153-48b1-4d80-81a9-f47c48372006 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:41.104685 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0000b84a-5ede-4759-abe1-2fbc36b75bfe service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:41.107654 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a3d1c6-728c-46fb-97d5-fc7d898bf006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:41.108562 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a3d1c6-728c-46fb-97d5-fc7d898bf006 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:41.108850 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a3d1c6-728c-46fb-97d5-fc7d898bf006 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:41.109172 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a3d1c6-728c-46fb-97d5-fc7d898bf006 admin admin] PATH_INFO: `/v3/projects/16078c76f554419992faa7570fe5aba1/users/42ade772e11a4725928ab12d3bccc4a7/roles/61b3b93d37a047e4bb8369b46fe20db9` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:41.115786 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 806/1604] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:32:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:32:41.117021 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f094cb34-d771-4262-abf6-6d31729bf291 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:41.117344 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f094cb34-d771-4262-abf6-6d31729bf291 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:41.117625 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f094cb34-d771-4262-abf6-6d31729bf291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:41.128050 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a3d1c6-728c-46fb-97d5-fc7d898bf006 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16078c76f554419992faa7570fe5aba1, user_id=42ade772e11a4725928ab12d3bccc4a7, role_id=61b3b93d37a047e4bb8369b46fe20db9)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:41.132523 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a3d1c6-728c-46fb-97d5-fc7d898bf006 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:41.135242 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:32:41.148306 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-f094cb34-d771-4262-abf6-6d31729bf291 None None] MFA Rules not processed for user `f9fa23907107401ebf143f7e550ae149`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:32:41.149394 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 799/1605] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:32:41 2022] PUT /identity/v3/projects/16078c76f554419992faa7570fe5aba1/users/42ade772e11a4725928ab12d3bccc4a7/roles/61b3b93d37a047e4bb8369b46fe20db9 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:32:41.150201 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a3d1c6-728c-46fb-97d5-fc7d898bf006 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:41.153407 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-f094cb34-d771-4262-abf6-6d31729bf291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:32:41.222919 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a3d1c6-728c-46fb-97d5-fc7d898bf006 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:41.228351 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e867b5ce-353a-45a6-ac1b-de3efd2ac573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:41.229158 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 807/1606] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:32:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:32:41.229307 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e867b5ce-353a-45a6-ac1b-de3efd2ac573 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:41.229754 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e867b5ce-353a-45a6-ac1b-de3efd2ac573 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:41.230091 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f094cb34-d771-4262-abf6-6d31729bf291 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:41.230219 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e867b5ce-353a-45a6-ac1b-de3efd2ac573 admin admin] PATH_INFO: `/v3/projects/fe39785b9222486eb49e696b67bb2f1f/users/fc49b67762b54904926edc482ccc6505/roles/61b3b93d37a047e4bb8369b46fe20db9` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:41.244012 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f094cb34-d771-4262-abf6-6d31729bf291 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:41.249889 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfb5e7f-1886-4b93-908a-21082bfce6d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:41.251586 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfb5e7f-1886-4b93-908a-21082bfce6d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:41.251586 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfb5e7f-1886-4b93-908a-21082bfce6d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:41.251751 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfb5e7f-1886-4b93-908a-21082bfce6d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:41.252377 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfb5e7f-1886-4b93-908a-21082bfce6d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:41.259821 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfb5e7f-1886-4b93-908a-21082bfce6d9 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:41.260552 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e867b5ce-353a-45a6-ac1b-de3efd2ac573 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe39785b9222486eb49e696b67bb2f1f, user_id=fc49b67762b54904926edc482ccc6505, role_id=61b3b93d37a047e4bb8369b46fe20db9)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:41.265409 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e867b5ce-353a-45a6-ac1b-de3efd2ac573 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:41.273009 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 808/1607] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:32:41 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:32:41.277588 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3915c66d-f363-4cb4-a7b3-5e3f1bed187a None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:41.277878 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3915c66d-f363-4cb4-a7b3-5e3f1bed187a None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:41.278598 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3915c66d-f363-4cb4-a7b3-5e3f1bed187a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:41.287358 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 800/1608] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:32:41 2022] PUT /identity/v3/projects/fe39785b9222486eb49e696b67bb2f1f/users/fc49b67762b54904926edc482ccc6505/roles/61b3b93d37a047e4bb8369b46fe20db9 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:32:41.288512 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e867b5ce-353a-45a6-ac1b-de3efd2ac573 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:41.326118 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-3915c66d-f363-4cb4-a7b3-5e3f1bed187a None None] MFA Rules not processed for user `d588a87dd3bc4a97b75f06491445a7a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:32:41.328330 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-3915c66d-f363-4cb4-a7b3-5e3f1bed187a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:32:41.374726 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e867b5ce-353a-45a6-ac1b-de3efd2ac573 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:41.380630 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12dba93-0245-40a5-81c6-fb27695b150f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:41.381371 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12dba93-0245-40a5-81c6-fb27695b150f admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:41.381721 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12dba93-0245-40a5-81c6-fb27695b150f admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:41.382013 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12dba93-0245-40a5-81c6-fb27695b150f admin admin] PATH_INFO: `/v3/projects/4bd83104ff2241519bcdd4e232eb6895/users/2455ea8abba547669fe22d2065a726c6/roles/61b3b93d37a047e4bb8369b46fe20db9` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:41.407073 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12dba93-0245-40a5-81c6-fb27695b150f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bd83104ff2241519bcdd4e232eb6895, user_id=2455ea8abba547669fe22d2065a726c6, role_id=61b3b93d37a047e4bb8369b46fe20db9)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:41.410411 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12dba93-0245-40a5-81c6-fb27695b150f admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:41.415414 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 809/1609] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:32:41 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:32:41.415414 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3915c66d-f363-4cb4-a7b3-5e3f1bed187a None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:41.431096 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 801/1610] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:32:41 2022] PUT /identity/v3/projects/4bd83104ff2241519bcdd4e232eb6895/users/2455ea8abba547669fe22d2065a726c6/roles/61b3b93d37a047e4bb8369b46fe20db9 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:32:41.432391 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9f0bd3-fbda-4685-94db-14c5caa43c4d None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:41.432687 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9f0bd3-fbda-4685-94db-14c5caa43c4d None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:41.432967 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9f0bd3-fbda-4685-94db-14c5caa43c4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:41.433327 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3915c66d-f363-4cb4-a7b3-5e3f1bed187a None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:41.455601 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-0d9f0bd3-fbda-4685-94db-14c5caa43c4d None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:32:41.457418 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-0d9f0bd3-fbda-4685-94db-14c5caa43c4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:32:41.513377 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d99223b-9c0f-4dc0-b60b-32da9643ed51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:41.514388 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d99223b-9c0f-4dc0-b60b-32da9643ed51 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:41.515015 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d99223b-9c0f-4dc0-b60b-32da9643ed51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:41.515015 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d99223b-9c0f-4dc0-b60b-32da9643ed51 admin admin] PATH_INFO: `/v3/projects/5be6efff0f0d482180e74f1cd075b592/users/37c704ce557c4f309cf00ef32646235f/roles/61b3b93d37a047e4bb8369b46fe20db9` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:41.520706 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:32:41.529931 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:32:41.535957 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d99223b-9c0f-4dc0-b60b-32da9643ed51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5be6efff0f0d482180e74f1cd075b592, user_id=37c704ce557c4f309cf00ef32646235f, role_id=61b3b93d37a047e4bb8369b46fe20db9)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:41.537689 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: WARNING py.warnings [None req-2d99223b-9c0f-4dc0-b60b-32da9643ed51 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 06 01:32:41.537689 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: warnings.warn(msg) Aug 06 01:32:41.537689 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: Aug 06 01:32:41.542242 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d99223b-9c0f-4dc0-b60b-32da9643ed51 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:41.542436 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 802/1611] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:32:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:32:41.544017 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baef5db-f56d-4816-8821-fd0539337b5c None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:41.544481 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baef5db-f56d-4816-8821-fd0539337b5c None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:41.550042 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baef5db-f56d-4816-8821-fd0539337b5c None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:41.552343 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 803/1612] 10.222.0.26 () {54 vars in 962 bytes} [Sat Aug 6 01:32:41 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:32:41.553423 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baef5db-f56d-4816-8821-fd0539337b5c None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:41.568882 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 810/1613] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:32:41 2022] PUT /identity/v3/projects/5be6efff0f0d482180e74f1cd075b592/users/37c704ce557c4f309cf00ef32646235f/roles/61b3b93d37a047e4bb8369b46fe20db9 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:32:41.569401 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baef5db-f56d-4816-8821-fd0539337b5c None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:41.570522 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a69d2e1-0299-4756-8480-741385d06224 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:41.570899 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a69d2e1-0299-4756-8480-741385d06224 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:41.571447 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a69d2e1-0299-4756-8480-741385d06224 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:41.604326 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-2a69d2e1-0299-4756-8480-741385d06224 None None] MFA Rules not processed for user `fc49b67762b54904926edc482ccc6505`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:32:41.605609 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-2a69d2e1-0299-4756-8480-741385d06224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:32:41.638464 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca32041e-c019-48e2-a505-28961e81be82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:41.639186 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca32041e-c019-48e2-a505-28961e81be82 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:41.639482 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca32041e-c019-48e2-a505-28961e81be82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:41.639777 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca32041e-c019-48e2-a505-28961e81be82 admin admin] PATH_INFO: `/v3/projects/a63fdb77fdfa4bd5943a72ffd59c7ce9/users/cbbb0d2c4aa4493a8ba41756903cd94c/roles/4559b9a0837249f895a13c70096f8d38` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:41.656226 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca32041e-c019-48e2-a505-28961e81be82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a63fdb77fdfa4bd5943a72ffd59c7ce9, user_id=cbbb0d2c4aa4493a8ba41756903cd94c, role_id=4559b9a0837249f895a13c70096f8d38)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:41.659078 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca32041e-c019-48e2-a505-28961e81be82 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:41.673951 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 804/1614] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:32:41 2022] PUT /identity/v3/projects/a63fdb77fdfa4bd5943a72ffd59c7ce9/users/cbbb0d2c4aa4493a8ba41756903cd94c/roles/4559b9a0837249f895a13c70096f8d38 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:32:41.674591 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca32041e-c019-48e2-a505-28961e81be82 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:41.680794 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 811/1615] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:32:41 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:32:41.682550 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682b1723-760f-4f5a-a223-8f6f3b624311 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:41.682847 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682b1723-760f-4f5a-a223-8f6f3b624311 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:41.683126 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682b1723-760f-4f5a-a223-8f6f3b624311 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:41.704404 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-682b1723-760f-4f5a-a223-8f6f3b624311 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:32:41.705627 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-682b1723-760f-4f5a-a223-8f6f3b624311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:32:41.749085 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca32041e-c019-48e2-a505-28961e81be82 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:41.753004 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbb6dae-c17c-4960-85dd-a46f435ac952 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1be1e6173e374a60b9e1fbf51a1ab571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:41.754455 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbb6dae-c17c-4960-85dd-a46f435ac952 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:41.754821 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbb6dae-c17c-4960-85dd-a46f435ac952 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:41.755331 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbb6dae-c17c-4960-85dd-a46f435ac952 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:41.772364 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 812/1616] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:32:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:32:41.774410 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca73aeb8-9a7f-4ed9-b2c3-229c942cf203 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:41.774410 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca73aeb8-9a7f-4ed9-b2c3-229c942cf203 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:41.774706 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca73aeb8-9a7f-4ed9-b2c3-229c942cf203 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:41.840395 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-ca73aeb8-9a7f-4ed9-b2c3-229c942cf203 None None] MFA Rules not processed for user `b87adc1d5bc240e99f95f6595b350be4`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:32:41.841663 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-ca73aeb8-9a7f-4ed9-b2c3-229c942cf203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:32:41.846436 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbb6dae-c17c-4960-85dd-a46f435ac952 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:41.849029 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbb6dae-c17c-4960-85dd-a46f435ac952 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:41.866272 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 805/1617] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:32:41 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:32:41.867088 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbb6dae-c17c-4960-85dd-a46f435ac952 service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:41.869495 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-8bbb6dae-c17c-4960-85dd-a46f435ac952 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:32:41.933217 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:32:41.940531 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:32:41.958579 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbb6dae-c17c-4960-85dd-a46f435ac952 service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:41.963446 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 813/1618] 10.222.0.26 () {58 vars in 1059 bytes} [Sat Aug 6 01:32:41 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:32:41.964891 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979346b0-4c11-4909-b7f5-cc73881939ac None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:41.965274 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979346b0-4c11-4909-b7f5-cc73881939ac None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:41.965593 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd4f4c2-ba02-48b4-8f5b-319c260435c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:41.965746 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979346b0-4c11-4909-b7f5-cc73881939ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:41.966821 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd4f4c2-ba02-48b4-8f5b-319c260435c0 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:41.967269 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd4f4c2-ba02-48b4-8f5b-319c260435c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:41.967702 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd4f4c2-ba02-48b4-8f5b-319c260435c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:41.978970 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd4f4c2-ba02-48b4-8f5b-319c260435c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:41.980744 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd4f4c2-ba02-48b4-8f5b-319c260435c0 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:41.989643 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-979346b0-4c11-4909-b7f5-cc73881939ac None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:32:41.991972 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-979346b0-4c11-4909-b7f5-cc73881939ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:32:41.997568 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 806/1619] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:32:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:32:41.999032 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd4f4c2-ba02-48b4-8f5b-319c260435c0 service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:42.000987 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 814/1620] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:32:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:32:42.001955 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979346b0-4c11-4909-b7f5-cc73881939ac None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:42.014427 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd4f4c2-ba02-48b4-8f5b-319c260435c0 service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:42.020241 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979346b0-4c11-4909-b7f5-cc73881939ac None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:42.020797 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bd80c7-bc49-4400-ab18-0b773cdbc066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:42.021502 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bd80c7-bc49-4400-ab18-0b773cdbc066 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:42.021843 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bd80c7-bc49-4400-ab18-0b773cdbc066 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:42.022123 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bd80c7-bc49-4400-ab18-0b773cdbc066 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:42.023495 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bd80c7-bc49-4400-ab18-0b773cdbc066 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:42.025845 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bd80c7-bc49-4400-ab18-0b773cdbc066 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:42.026548 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fb0d8f-f15f-4cf1-b876-41a205191312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:42.027102 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fb0d8f-f15f-4cf1-b876-41a205191312 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:42.027450 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fb0d8f-f15f-4cf1-b876-41a205191312 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:42.027791 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fb0d8f-f15f-4cf1-b876-41a205191312 admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:42.028626 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fb0d8f-f15f-4cf1-b876-41a205191312 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:42.031869 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fb0d8f-f15f-4cf1-b876-41a205191312 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:42.044728 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 807/1621] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:32:41 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:32:42.044728 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bd80c7-bc49-4400-ab18-0b773cdbc066 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:42.058761 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bd80c7-bc49-4400-ab18-0b773cdbc066 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:42.064702 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1842a2d9-4889-468d-898f-84c500572115 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:42.065481 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1842a2d9-4889-468d-898f-84c500572115 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:42.066421 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1842a2d9-4889-468d-898f-84c500572115 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:42.066966 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1842a2d9-4889-468d-898f-84c500572115 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:42.068650 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 815/1622] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Aug 6 01:32:42 2022] POST /identity/v3/projects => generated 363 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:32:42.069328 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fb0d8f-f15f-4cf1-b876-41a205191312 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:42.088287 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1842a2d9-4889-468d-898f-84c500572115 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:42.089902 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1842a2d9-4889-468d-898f-84c500572115 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:42.105250 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 808/1623] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:32:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:32:42.106408 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1842a2d9-4889-468d-898f-84c500572115 service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:42.128808 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1842a2d9-4889-468d-898f-84c500572115 service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:42.197287 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fb0d8f-f15f-4cf1-b876-41a205191312 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:42.207487 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb51078-b150-47e4-8964-ef9155c8ea4a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87adc1d5bc240e99f95f6595b350be4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:42.208457 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb51078-b150-47e4-8964-ef9155c8ea4a service nova] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:42.208803 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb51078-b150-47e4-8964-ef9155c8ea4a service nova] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:42.209205 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb51078-b150-47e4-8964-ef9155c8ea4a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:42.244944 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c69994b-3a02-4301-b29a-88b3259e1355 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:42.245613 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c69994b-3a02-4301-b29a-88b3259e1355 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:42.245898 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c69994b-3a02-4301-b29a-88b3259e1355 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:42.246175 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c69994b-3a02-4301-b29a-88b3259e1355 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:42.301080 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb51078-b150-47e4-8964-ef9155c8ea4a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:42.302756 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb51078-b150-47e4-8964-ef9155c8ea4a service nova] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:42.319987 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 816/1624] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:32:42 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:32:42.321263 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb51078-b150-47e4-8964-ef9155c8ea4a service nova] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:42.383143 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c69994b-3a02-4301-b29a-88b3259e1355 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:42.388130 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c69994b-3a02-4301-b29a-88b3259e1355 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:42.410953 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 809/1625] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:32:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:32:42.427819 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c69994b-3a02-4301-b29a-88b3259e1355 service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:42.465452 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c69994b-3a02-4301-b29a-88b3259e1355 service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:42.488656 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b38a64-b55b-4056-a667-569e39d05134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:42.490234 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b38a64-b55b-4056-a667-569e39d05134 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:42.490590 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b38a64-b55b-4056-a667-569e39d05134 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:42.491649 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb51078-b150-47e4-8964-ef9155c8ea4a service nova] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:42.492023 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b38a64-b55b-4056-a667-569e39d05134 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:42.494240 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b38a64-b55b-4056-a667-569e39d05134 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:42.503080 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b38a64-b55b-4056-a667-569e39d05134 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:42.517297 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798d9f20-9632-4b84-a30c-7a4fdac2e5ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:42.523001 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798d9f20-9632-4b84-a30c-7a4fdac2e5ca admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:42.524085 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798d9f20-9632-4b84-a30c-7a4fdac2e5ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:42.525130 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798d9f20-9632-4b84-a30c-7a4fdac2e5ca admin admin] PATH_INFO: `/v3/projects/a63fdb77fdfa4bd5943a72ffd59c7ce9/users/cbbb0d2c4aa4493a8ba41756903cd94c/roles/4559b9a0837249f895a13c70096f8d38` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:42.530884 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798d9f20-9632-4b84-a30c-7a4fdac2e5ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=a63fdb77fdfa4bd5943a72ffd59c7ce9, user_id=cbbb0d2c4aa4493a8ba41756903cd94c, role_id=4559b9a0837249f895a13c70096f8d38)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:42.536498 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798d9f20-9632-4b84-a30c-7a4fdac2e5ca admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:42.555103 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:32:42.603348 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 810/1626] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:32:42 2022] POST /identity/v3/users => generated 288 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:32:42.604741 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b38a64-b55b-4056-a667-569e39d05134 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:42.608829 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 817/1627] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:32:42 2022] PUT /identity/v3/projects/a63fdb77fdfa4bd5943a72ffd59c7ce9/users/cbbb0d2c4aa4493a8ba41756903cd94c/roles/4559b9a0837249f895a13c70096f8d38 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:32:42.619910 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26105a25-7157-40be-a2b9-e86349d0e66a None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:42.620364 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26105a25-7157-40be-a2b9-e86349d0e66a None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:42.620930 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26105a25-7157-40be-a2b9-e86349d0e66a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:42.635794 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b38a64-b55b-4056-a667-569e39d05134 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:42.711831 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-26105a25-7157-40be-a2b9-e86349d0e66a None None] MFA Rules not processed for user `cbbb0d2c4aa4493a8ba41756903cd94c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:32:42.713231 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-26105a25-7157-40be-a2b9-e86349d0e66a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:32:42.797126 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:32:42.838560 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a94add-7ce1-45d7-a33f-460c138fcf2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:42.839348 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a94add-7ce1-45d7-a33f-460c138fcf2b admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:42.839569 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a94add-7ce1-45d7-a33f-460c138fcf2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:42.839926 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a94add-7ce1-45d7-a33f-460c138fcf2b admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:42.840520 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a94add-7ce1-45d7-a33f-460c138fcf2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:42.842126 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a94add-7ce1-45d7-a33f-460c138fcf2b admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:42.872045 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 811/1628] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:32:42 2022] GET /identity/v3/roles => generated 1420 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:32:42.872945 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a94add-7ce1-45d7-a33f-460c138fcf2b admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:42.893315 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a94add-7ce1-45d7-a33f-460c138fcf2b admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:42.908625 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca0faf3-a710-4d09-bf69-d14426680f27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:42.910351 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca0faf3-a710-4d09-bf69-d14426680f27 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:42.910792 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca0faf3-a710-4d09-bf69-d14426680f27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:42.911510 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca0faf3-a710-4d09-bf69-d14426680f27 admin admin] PATH_INFO: `/v3/projects/2cbf095db09e439a804134181720a49f/users/c25ff9353575471398101a913a863919/roles/4559b9a0837249f895a13c70096f8d38` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:42.941540 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 818/1629] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:32:42 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:32:42.951644 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26105a25-7157-40be-a2b9-e86349d0e66a None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:42.973601 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca0faf3-a710-4d09-bf69-d14426680f27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2cbf095db09e439a804134181720a49f, user_id=c25ff9353575471398101a913a863919, role_id=4559b9a0837249f895a13c70096f8d38)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:42.985354 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca0faf3-a710-4d09-bf69-d14426680f27 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:43.041673 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 812/1630] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:32:42 2022] PUT /identity/v3/projects/2cbf095db09e439a804134181720a49f/users/c25ff9353575471398101a913a863919/roles/4559b9a0837249f895a13c70096f8d38 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:32:43.045658 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca0faf3-a710-4d09-bf69-d14426680f27 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:43.117848 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26105a25-7157-40be-a2b9-e86349d0e66a None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:43.207832 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca0faf3-a710-4d09-bf69-d14426680f27 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:43.221567 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067878d3-4d78-44c1-b105-5c332a72b32d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:43.222760 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067878d3-4d78-44c1-b105-5c332a72b32d admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:43.223251 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067878d3-4d78-44c1-b105-5c332a72b32d admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:43.223541 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067878d3-4d78-44c1-b105-5c332a72b32d admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:43.224252 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067878d3-4d78-44c1-b105-5c332a72b32d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:43.228838 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067878d3-4d78-44c1-b105-5c332a72b32d admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:43.254083 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 813/1631] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:32:43 2022] GET /identity/v3/roles => generated 1420 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 01:32:43.255111 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061c9197-3bab-4597-a8ae-fe7ecd77bf4c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1be1e6173e374a60b9e1fbf51a1ab571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:43.256050 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061c9197-3bab-4597-a8ae-fe7ecd77bf4c service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:43.256343 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061c9197-3bab-4597-a8ae-fe7ecd77bf4c service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:43.256632 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061c9197-3bab-4597-a8ae-fe7ecd77bf4c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:43.259909 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067878d3-4d78-44c1-b105-5c332a72b32d admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:43.300990 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067878d3-4d78-44c1-b105-5c332a72b32d admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:43.317316 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94304863-61c3-4c3b-b175-5228fa2735ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:43.318013 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94304863-61c3-4c3b-b175-5228fa2735ef admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:43.318300 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94304863-61c3-4c3b-b175-5228fa2735ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:43.318875 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94304863-61c3-4c3b-b175-5228fa2735ef admin admin] PATH_INFO: `/v3/projects/2cbf095db09e439a804134181720a49f/users/c25ff9353575471398101a913a863919/roles/4559b9a0837249f895a13c70096f8d38` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:43.324225 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94304863-61c3-4c3b-b175-5228fa2735ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=2cbf095db09e439a804134181720a49f, user_id=c25ff9353575471398101a913a863919, role_id=4559b9a0837249f895a13c70096f8d38)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:43.328111 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94304863-61c3-4c3b-b175-5228fa2735ef admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:43.356938 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 814/1632] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:32:43 2022] PUT /identity/v3/projects/2cbf095db09e439a804134181720a49f/users/c25ff9353575471398101a913a863919/roles/4559b9a0837249f895a13c70096f8d38 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:32:43.368454 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8b7509-e253-40af-9139-386641c486f0 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:43.368743 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8b7509-e253-40af-9139-386641c486f0 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:43.369029 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8b7509-e253-40af-9139-386641c486f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:43.390814 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:32:43.464588 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-4a8b7509-e253-40af-9139-386641c486f0 None None] MFA Rules not processed for user `c25ff9353575471398101a913a863919`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:32:43.464588 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-4a8b7509-e253-40af-9139-386641c486f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:32:43.485169 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061c9197-3bab-4597-a8ae-fe7ecd77bf4c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:43.485169 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061c9197-3bab-4597-a8ae-fe7ecd77bf4c service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:43.508137 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 819/1633] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:32:42 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 554 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:32:43.633224 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac49d5c-fdc4-4067-8f3b-e617566c3790 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:43.633611 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac49d5c-fdc4-4067-8f3b-e617566c3790 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:43.634044 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac49d5c-fdc4-4067-8f3b-e617566c3790 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:43.636360 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 820/1634] 10.222.0.26 () {54 vars in 962 bytes} [Sat Aug 6 01:32:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:32:43.642594 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c57e85-796a-47c6-b5d8-8a72ce447d78 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:43.643186 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c57e85-796a-47c6-b5d8-8a72ce447d78 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:43.643595 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c57e85-796a-47c6-b5d8-8a72ce447d78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:43.677299 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-84c57e85-796a-47c6-b5d8-8a72ce447d78 None None] MFA Rules not processed for user `b87adc1d5bc240e99f95f6595b350be4`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:32:43.681559 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-84c57e85-796a-47c6-b5d8-8a72ce447d78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:32:43.703462 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 815/1635] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:32:43 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 336 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:32:43.718587 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8b7509-e253-40af-9139-386641c486f0 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:43.761745 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:32:43.798680 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8b7509-e253-40af-9139-386641c486f0 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:43.803169 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6484a8a2-9338-42b2-8d41-f7b57237c0b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1be1e6173e374a60b9e1fbf51a1ab571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:43.803850 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6484a8a2-9338-42b2-8d41-f7b57237c0b6 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:43.804138 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6484a8a2-9338-42b2-8d41-f7b57237c0b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:43.804415 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6484a8a2-9338-42b2-8d41-f7b57237c0b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:43.819885 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6484a8a2-9338-42b2-8d41-f7b57237c0b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:43.821399 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6484a8a2-9338-42b2-8d41-f7b57237c0b6 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:43.834311 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 821/1636] 10.222.0.26 () {58 vars in 1059 bytes} [Sat Aug 6 01:32:43 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:32:43.835633 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c57e85-796a-47c6-b5d8-8a72ce447d78 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:43.850939 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 816/1637] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:32:43 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:32:43.854020 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999afea0-5064-462e-adf0-3954eba3e67a None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:43.855476 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999afea0-5064-462e-adf0-3954eba3e67a None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:43.855774 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999afea0-5064-462e-adf0-3954eba3e67a None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:43.860662 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 817/1638] 10.222.0.26 () {54 vars in 912 bytes} [Sat Aug 6 01:32:43 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:32:43.867306 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac6d42e-1428-49fc-8a03-18845e2f6b50 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:43.867694 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac6d42e-1428-49fc-8a03-18845e2f6b50 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:43.867983 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac6d42e-1428-49fc-8a03-18845e2f6b50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:43.937458 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-8ac6d42e-1428-49fc-8a03-18845e2f6b50 None None] MFA Rules not processed for user `b87adc1d5bc240e99f95f6595b350be4`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:32:43.939197 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-8ac6d42e-1428-49fc-8a03-18845e2f6b50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:32:43.964327 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 818/1639] 10.222.0.26 () {58 vars in 1009 bytes} [Sat Aug 6 01:32:43 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:32:43.977251 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac6d42e-1428-49fc-8a03-18845e2f6b50 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:43.992817 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac6d42e-1428-49fc-8a03-18845e2f6b50 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:44.006226 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7fa38a-a3f1-4a0d-aa29-446d57268577 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1be1e6173e374a60b9e1fbf51a1ab571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:44.008419 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7fa38a-a3f1-4a0d-aa29-446d57268577 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:44.008713 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7fa38a-a3f1-4a0d-aa29-446d57268577 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:44.010767 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7fa38a-a3f1-4a0d-aa29-446d57268577 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:44.031163 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c57e85-796a-47c6-b5d8-8a72ce447d78 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:44.039111 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7fa38a-a3f1-4a0d-aa29-446d57268577 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:44.056362 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7fa38a-a3f1-4a0d-aa29-446d57268577 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:44.059898 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619f11b5-9cb3-48c3-a42b-fa05c868f531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:44.070374 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619f11b5-9cb3-48c3-a42b-fa05c868f531 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:44.070698 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619f11b5-9cb3-48c3-a42b-fa05c868f531 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:44.071107 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619f11b5-9cb3-48c3-a42b-fa05c868f531 admin admin] PATH_INFO: `/v3/users/bfb318d8b4d7478a92bcfac9a07a447f` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:44.074858 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619f11b5-9cb3-48c3-a42b-fa05c868f531 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bfb318d8b4d7478a92bcfac9a07a447f)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:44.076123 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: WARNING py.warnings [None req-619f11b5-9cb3-48c3-a42b-fa05c868f531 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 06 01:32:44.076123 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: warnings.warn(msg) Aug 06 01:32:44.076123 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: Aug 06 01:32:44.078312 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619f11b5-9cb3-48c3-a42b-fa05c868f531 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:44.087750 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 819/1640] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:32:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:32:44.091102 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7fa38a-a3f1-4a0d-aa29-446d57268577 service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:44.104741 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7fa38a-a3f1-4a0d-aa29-446d57268577 service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:44.110920 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90209fa-c97e-4b58-978d-0378cd36a471 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1be1e6173e374a60b9e1fbf51a1ab571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:44.112556 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90209fa-c97e-4b58-978d-0378cd36a471 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:44.114076 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90209fa-c97e-4b58-978d-0378cd36a471 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:44.114657 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90209fa-c97e-4b58-978d-0378cd36a471 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:44.245879 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90209fa-c97e-4b58-978d-0378cd36a471 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:44.247949 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90209fa-c97e-4b58-978d-0378cd36a471 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:44.271979 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 820/1641] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:32:44 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:32:44.274683 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c1112d-596b-407f-91f3-9e2d51bb576b None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:44.275325 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c1112d-596b-407f-91f3-9e2d51bb576b None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:44.276826 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-619f11b5-9cb3-48c3-a42b-fa05c868f531 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bfb318d8b4d7478a92bcfac9a07a447f'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:32:44.277387 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c1112d-596b-407f-91f3-9e2d51bb576b None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:44.281130 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 821/1642] 10.222.0.26 () {54 vars in 912 bytes} [Sat Aug 6 01:32:44 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:32:44.287286 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0183527a-3ccf-463a-9bb8-cd3202c015d9 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:44.288004 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0183527a-3ccf-463a-9bb8-cd3202c015d9 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:44.288625 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0183527a-3ccf-463a-9bb8-cd3202c015d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:44.292234 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-619f11b5-9cb3-48c3-a42b-fa05c868f531 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bfb318d8b4d7478a92bcfac9a07a447f'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:32:44.293251 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-619f11b5-9cb3-48c3-a42b-fa05c868f531 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bfb318d8b4d7478a92bcfac9a07a447f'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:32:44.293879 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-619f11b5-9cb3-48c3-a42b-fa05c868f531 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bfb318d8b4d7478a92bcfac9a07a447f'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:32:44.319843 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-619f11b5-9cb3-48c3-a42b-fa05c868f531 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bfb318d8b4d7478a92bcfac9a07a447f'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:32:44.334306 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:32:44.349645 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-0183527a-3ccf-463a-9bb8-cd3202c015d9 None None] MFA Rules not processed for user `b87adc1d5bc240e99f95f6595b350be4`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:32:44.352771 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-0183527a-3ccf-463a-9bb8-cd3202c015d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:32:44.353759 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 822/1643] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Aug 6 01:32:43 2022] DELETE /identity/v3/users/bfb318d8b4d7478a92bcfac9a07a447f => generated 0 bytes in 511 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:32:44.355697 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2554328d-0573-4481-ab31-e2c4a7937823 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:44.356218 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2554328d-0573-4481-ab31-e2c4a7937823 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:44.356629 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2554328d-0573-4481-ab31-e2c4a7937823 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:44.391848 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-2554328d-0573-4481-ab31-e2c4a7937823 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:32:44.393403 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-2554328d-0573-4481-ab31-e2c4a7937823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:32:44.521333 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 823/1644] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:32:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:32:44.536582 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 822/1645] 10.222.0.26 () {58 vars in 1009 bytes} [Sat Aug 6 01:32:44 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:32:44.554452 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2554328d-0573-4481-ab31-e2c4a7937823 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:44.554678 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0183527a-3ccf-463a-9bb8-cd3202c015d9 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:44.556323 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-0183527a-3ccf-463a-9bb8-cd3202c015d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:32:44.560640 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-2554328d-0573-4481-ab31-e2c4a7937823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:32:44.682182 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2554328d-0573-4481-ab31-e2c4a7937823 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:44.696157 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2600f938-4077-45ff-8f9e-475bb92a0c80 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1be1e6173e374a60b9e1fbf51a1ab571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:44.697070 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2600f938-4077-45ff-8f9e-475bb92a0c80 service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:44.697817 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2600f938-4077-45ff-8f9e-475bb92a0c80 service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:44.698711 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2600f938-4077-45ff-8f9e-475bb92a0c80 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:44.714638 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:32:44.715542 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2600f938-4077-45ff-8f9e-475bb92a0c80 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:44.718010 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2600f938-4077-45ff-8f9e-475bb92a0c80 service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:44.739683 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 824/1646] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:32:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:32:44.744162 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0183527a-3ccf-463a-9bb8-cd3202c015d9 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:44.752016 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d4c149-35b6-4888-b63f-77107ba98767 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:44.755099 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d4c149-35b6-4888-b63f-77107ba98767 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:44.755628 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d4c149-35b6-4888-b63f-77107ba98767 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:44.756115 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d4c149-35b6-4888-b63f-77107ba98767 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:44.772055 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d4c149-35b6-4888-b63f-77107ba98767 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:44.774121 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d4c149-35b6-4888-b63f-77107ba98767 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:44.801991 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 823/1647] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:32:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:32:44.901397 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2600f938-4077-45ff-8f9e-475bb92a0c80 service cinder] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:44.902920 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-2600f938-4077-45ff-8f9e-475bb92a0c80 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:32:44.919581 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2600f938-4077-45ff-8f9e-475bb92a0c80 service cinder] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:44.926776 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88b6f99-7bf4-4fe0-8443-ecfbe1e45146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:44.927585 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88b6f99-7bf4-4fe0-8443-ecfbe1e45146 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:44.927933 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88b6f99-7bf4-4fe0-8443-ecfbe1e45146 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:44.928213 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88b6f99-7bf4-4fe0-8443-ecfbe1e45146 admin admin] PATH_INFO: `/v3/users/f9fa23907107401ebf143f7e550ae149` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:44.929492 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88b6f99-7bf4-4fe0-8443-ecfbe1e45146 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9fa23907107401ebf143f7e550ae149)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:44.931510 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88b6f99-7bf4-4fe0-8443-ecfbe1e45146 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:45.004183 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-d88b6f99-7bf4-4fe0-8443-ecfbe1e45146 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9fa23907107401ebf143f7e550ae149'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:32:45.013001 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-d88b6f99-7bf4-4fe0-8443-ecfbe1e45146 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9fa23907107401ebf143f7e550ae149'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:32:45.013754 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-d88b6f99-7bf4-4fe0-8443-ecfbe1e45146 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9fa23907107401ebf143f7e550ae149'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:32:45.014680 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-d88b6f99-7bf4-4fe0-8443-ecfbe1e45146 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9fa23907107401ebf143f7e550ae149'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:32:45.040014 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-d88b6f99-7bf4-4fe0-8443-ecfbe1e45146 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9fa23907107401ebf143f7e550ae149'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:32:45.071350 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 825/1648] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Aug 6 01:32:44 2022] DELETE /identity/v3/users/f9fa23907107401ebf143f7e550ae149 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:32:45.079766 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba07096-958e-48bc-ad26-5e811f628b8a None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:45.080311 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba07096-958e-48bc-ad26-5e811f628b8a None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:45.080311 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba07096-958e-48bc-ad26-5e811f628b8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:45.116055 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-1ba07096-958e-48bc-ad26-5e811f628b8a None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:32:45.117965 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-1ba07096-958e-48bc-ad26-5e811f628b8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:32:45.273625 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 824/1649] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:32:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:32:45.305092 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88b6f99-7bf4-4fe0-8443-ecfbe1e45146 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:45.308701 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-d88b6f99-7bf4-4fe0-8443-ecfbe1e45146 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:32:45.342692 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:32:45.411947 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88b6f99-7bf4-4fe0-8443-ecfbe1e45146 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:45.416367 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3589c4db-cd91-4612-88bb-80b52473de6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:45.417112 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3589c4db-cd91-4612-88bb-80b52473de6a service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:45.417593 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3589c4db-cd91-4612-88bb-80b52473de6a service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:45.417947 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3589c4db-cd91-4612-88bb-80b52473de6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:45.432877 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3589c4db-cd91-4612-88bb-80b52473de6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:45.434740 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3589c4db-cd91-4612-88bb-80b52473de6a service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:45.450081 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 826/1650] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:32:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:32:46.523743 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba07096-958e-48bc-ad26-5e811f628b8a None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:46.527202 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-1ba07096-958e-48bc-ad26-5e811f628b8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:32:46.555163 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba07096-958e-48bc-ad26-5e811f628b8a None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:46.572809 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c58c1b-e16c-42b9-a63f-72421b09fde7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:46.574466 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c58c1b-e16c-42b9-a63f-72421b09fde7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:46.574748 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c58c1b-e16c-42b9-a63f-72421b09fde7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:46.575023 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c58c1b-e16c-42b9-a63f-72421b09fde7 admin admin] PATH_INFO: `/v3/projects/a4a60a35d1b246babe9832dc26bb4450` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:46.581707 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c58c1b-e16c-42b9-a63f-72421b09fde7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4a60a35d1b246babe9832dc26bb4450)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:46.586808 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: WARNING py.warnings [None req-96c58c1b-e16c-42b9-a63f-72421b09fde7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 06 01:32:46.586808 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: warnings.warn(msg) Aug 06 01:32:46.586808 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: Aug 06 01:32:46.598791 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c58c1b-e16c-42b9-a63f-72421b09fde7 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:46.740380 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-96c58c1b-e16c-42b9-a63f-72421b09fde7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4a60a35d1b246babe9832dc26bb4450', 'request_id': 'req-96c58c1b-e16c-42b9-a63f-72421b09fde7'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:32:46.773363 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-96c58c1b-e16c-42b9-a63f-72421b09fde7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4a60a35d1b246babe9832dc26bb4450', 'request_id': 'req-96c58c1b-e16c-42b9-a63f-72421b09fde7'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:32:46.805534 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-96c58c1b-e16c-42b9-a63f-72421b09fde7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4a60a35d1b246babe9832dc26bb4450', 'request_id': 'req-96c58c1b-e16c-42b9-a63f-72421b09fde7'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:32:46.839257 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-96c58c1b-e16c-42b9-a63f-72421b09fde7 admin admin] The token cache is being invalidate because project a4a60a35d1b246babe9832dc26bb4450 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:32:46.840308 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-96c58c1b-e16c-42b9-a63f-72421b09fde7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:32:46.842270 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-96c58c1b-e16c-42b9-a63f-72421b09fde7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:32:46.844108 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 825/1651] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Aug 6 01:32:46 2022] DELETE /identity/v3/projects/a4a60a35d1b246babe9832dc26bb4450 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:32:46.852550 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a423d7b8-f647-415d-b2df-7981d406b34d None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:46.853142 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a423d7b8-f647-415d-b2df-7981d406b34d None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:46.853142 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a423d7b8-f647-415d-b2df-7981d406b34d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:46.926751 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-a423d7b8-f647-415d-b2df-7981d406b34d None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:32:46.927980 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-a423d7b8-f647-415d-b2df-7981d406b34d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:32:46.961018 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c58c1b-e16c-42b9-a63f-72421b09fde7 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:46.964146 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-96c58c1b-e16c-42b9-a63f-72421b09fde7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:32:47.074691 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 827/1652] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:32:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:32:47.084270 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8368e6-f21e-43d6-9d49-5c8eaf728e41 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:47.084783 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8368e6-f21e-43d6-9d49-5c8eaf728e41 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:47.085248 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8368e6-f21e-43d6-9d49-5c8eaf728e41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:47.120318 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c58c1b-e16c-42b9-a63f-72421b09fde7 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:47.143709 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5390332f-00d2-45dd-b8d3-43d620c359eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:47.147662 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5390332f-00d2-45dd-b8d3-43d620c359eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:47.147959 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5390332f-00d2-45dd-b8d3-43d620c359eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:47.148245 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5390332f-00d2-45dd-b8d3-43d620c359eb admin admin] PATH_INFO: `/v3/projects/08eb0e0a70fb4b7baf477606bc46f603` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:47.149639 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5390332f-00d2-45dd-b8d3-43d620c359eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=08eb0e0a70fb4b7baf477606bc46f603)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:47.155141 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5390332f-00d2-45dd-b8d3-43d620c359eb admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:47.155422 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-ae8368e6-f21e-43d6-9d49-5c8eaf728e41 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:32:47.156717 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-ae8368e6-f21e-43d6-9d49-5c8eaf728e41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:32:47.181024 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 828/1653] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:32:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:32:47.181840 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:32:47.182280 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8368e6-f21e-43d6-9d49-5c8eaf728e41 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:47.195922 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8368e6-f21e-43d6-9d49-5c8eaf728e41 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:47.225943 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea75f477-25fe-41c2-8dff-66633c681245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:47.226585 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea75f477-25fe-41c2-8dff-66633c681245 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:47.226864 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea75f477-25fe-41c2-8dff-66633c681245 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:47.227138 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea75f477-25fe-41c2-8dff-66633c681245 admin admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:47.227853 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea75f477-25fe-41c2-8dff-66633c681245 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:32:47.228155 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea75f477-25fe-41c2-8dff-66633c681245 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:47.229812 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea75f477-25fe-41c2-8dff-66633c681245 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:47.276358 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 829/1654] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Aug 6 01:32:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:32:47.277974 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea75f477-25fe-41c2-8dff-66633c681245 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:47.303721 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea75f477-25fe-41c2-8dff-66633c681245 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:47.311048 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-5390332f-00d2-45dd-b8d3-43d620c359eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08eb0e0a70fb4b7baf477606bc46f603', 'request_id': 'req-5390332f-00d2-45dd-b8d3-43d620c359eb'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:32:47.335863 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-5390332f-00d2-45dd-b8d3-43d620c359eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08eb0e0a70fb4b7baf477606bc46f603', 'request_id': 'req-5390332f-00d2-45dd-b8d3-43d620c359eb'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:32:47.360070 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-5390332f-00d2-45dd-b8d3-43d620c359eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08eb0e0a70fb4b7baf477606bc46f603', 'request_id': 'req-5390332f-00d2-45dd-b8d3-43d620c359eb'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:32:47.391854 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-5390332f-00d2-45dd-b8d3-43d620c359eb admin admin] The token cache is being invalidate because project 08eb0e0a70fb4b7baf477606bc46f603 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:32:47.392162 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-5390332f-00d2-45dd-b8d3-43d620c359eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:32:47.393323 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-5390332f-00d2-45dd-b8d3-43d620c359eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:32:47.396344 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 826/1655] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Aug 6 01:32:46 2022] DELETE /identity/v3/projects/08eb0e0a70fb4b7baf477606bc46f603 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:32:47.402198 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5390332f-00d2-45dd-b8d3-43d620c359eb admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:47.408656 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-5390332f-00d2-45dd-b8d3-43d620c359eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:32:47.458656 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b0c3e4-6cbf-45b9-b279-280d29518492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:47.460413 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b0c3e4-6cbf-45b9-b279-280d29518492 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:47.461356 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b0c3e4-6cbf-45b9-b279-280d29518492 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:47.464230 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b0c3e4-6cbf-45b9-b279-280d29518492 admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:47.465177 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b0c3e4-6cbf-45b9-b279-280d29518492 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:47.467248 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b0c3e4-6cbf-45b9-b279-280d29518492 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:47.551957 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 830/1656] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Aug 6 01:32:47 2022] POST /identity/v3/projects => generated 409 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:32:47.554102 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b0c3e4-6cbf-45b9-b279-280d29518492 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:47.557075 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-43b0c3e4-6cbf-45b9-b279-280d29518492 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:32:47.631995 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5390332f-00d2-45dd-b8d3-43d620c359eb admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:47.670407 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:32:47.729936 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd780f60-11f1-437b-94a1-5b8bf42f70c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:47.733190 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd780f60-11f1-437b-94a1-5b8bf42f70c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:47.734136 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd780f60-11f1-437b-94a1-5b8bf42f70c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:47.735035 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd780f60-11f1-437b-94a1-5b8bf42f70c7 admin admin] PATH_INFO: `/v3/users/cbbb0d2c4aa4493a8ba41756903cd94c` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:47.736579 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd780f60-11f1-437b-94a1-5b8bf42f70c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbbb0d2c4aa4493a8ba41756903cd94c)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:47.742839 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd780f60-11f1-437b-94a1-5b8bf42f70c7 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:47.773186 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b0c3e4-6cbf-45b9-b279-280d29518492 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:47.787231 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d77ce5d-0dd8-4f86-a5e3-d7c3b61b6c94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:47.789271 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d77ce5d-0dd8-4f86-a5e3-d7c3b61b6c94 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:47.789715 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d77ce5d-0dd8-4f86-a5e3-d7c3b61b6c94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:47.791175 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d77ce5d-0dd8-4f86-a5e3-d7c3b61b6c94 admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:47.791979 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d77ce5d-0dd8-4f86-a5e3-d7c3b61b6c94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:47.796834 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d77ce5d-0dd8-4f86-a5e3-d7c3b61b6c94 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:47.848330 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-bd780f60-11f1-437b-94a1-5b8bf42f70c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbbb0d2c4aa4493a8ba41756903cd94c'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:32:47.849826 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-bd780f60-11f1-437b-94a1-5b8bf42f70c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbbb0d2c4aa4493a8ba41756903cd94c'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:32:47.850487 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 831/1657] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:32:47 2022] POST /identity/v3/users => generated 305 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:32:47.852681 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-bd780f60-11f1-437b-94a1-5b8bf42f70c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbbb0d2c4aa4493a8ba41756903cd94c'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:32:47.854206 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:32:47.855592 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d77ce5d-0dd8-4f86-a5e3-d7c3b61b6c94 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:47.857978 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-5d77ce5d-0dd8-4f86-a5e3-d7c3b61b6c94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:32:47.885220 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-bd780f60-11f1-437b-94a1-5b8bf42f70c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbbb0d2c4aa4493a8ba41756903cd94c'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:32:47.891462 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d77ce5d-0dd8-4f86-a5e3-d7c3b61b6c94 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:47.901926 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-bd780f60-11f1-437b-94a1-5b8bf42f70c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbbb0d2c4aa4493a8ba41756903cd94c'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:32:47.902996 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cf9a28-4ac2-428b-bdcb-1dffb92a852e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:47.903642 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cf9a28-4ac2-428b-bdcb-1dffb92a852e admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:47.903927 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cf9a28-4ac2-428b-bdcb-1dffb92a852e admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:47.904205 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cf9a28-4ac2-428b-bdcb-1dffb92a852e admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:47.907846 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cf9a28-4ac2-428b-bdcb-1dffb92a852e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:47.911508 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cf9a28-4ac2-428b-bdcb-1dffb92a852e admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:47.930971 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 827/1658] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Aug 6 01:32:47 2022] DELETE /identity/v3/users/cbbb0d2c4aa4493a8ba41756903cd94c => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:32:47.945685 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 832/1659] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:32:47 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:32:47.949686 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd780f60-11f1-437b-94a1-5b8bf42f70c7 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:48.154929 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd780f60-11f1-437b-94a1-5b8bf42f70c7 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:48.160941 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee757d2-22f8-4a06-a036-076df8e6d289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:48.161741 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee757d2-22f8-4a06-a036-076df8e6d289 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:48.162027 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee757d2-22f8-4a06-a036-076df8e6d289 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:48.162304 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee757d2-22f8-4a06-a036-076df8e6d289 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:48.163391 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee757d2-22f8-4a06-a036-076df8e6d289 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:48.166492 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee757d2-22f8-4a06-a036-076df8e6d289 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:48.179429 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 828/1660] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:32:47 2022] GET /identity/v3/roles => generated 1420 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:32:48.183707 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cf9a28-4ac2-428b-bdcb-1dffb92a852e admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:48.199472 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cf9a28-4ac2-428b-bdcb-1dffb92a852e admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:48.219444 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39fb591-e636-4351-83b5-836d0a90b160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:48.225217 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39fb591-e636-4351-83b5-836d0a90b160 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:48.225572 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39fb591-e636-4351-83b5-836d0a90b160 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:48.228547 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39fb591-e636-4351-83b5-836d0a90b160 admin admin] PATH_INFO: `/v3/projects/28e80afa761e41989ae14fe250b3656c/users/a0d9868a89514f7590e7eb84915aa260/roles/61b3b93d37a047e4bb8369b46fe20db9` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:48.285029 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39fb591-e636-4351-83b5-836d0a90b160 admin admin] RBAC: Authorizing `identity:create_grant(project_id=28e80afa761e41989ae14fe250b3656c, user_id=a0d9868a89514f7590e7eb84915aa260, role_id=61b3b93d37a047e4bb8369b46fe20db9)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:48.289868 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39fb591-e636-4351-83b5-836d0a90b160 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:48.327815 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 833/1661] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:32:48 2022] PUT /identity/v3/projects/28e80afa761e41989ae14fe250b3656c/users/a0d9868a89514f7590e7eb84915aa260/roles/61b3b93d37a047e4bb8369b46fe20db9 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:32:48.329536 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8095716-cdd3-4f9e-b0da-298803d5f800 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:48.329833 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8095716-cdd3-4f9e-b0da-298803d5f800 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:48.330109 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8095716-cdd3-4f9e-b0da-298803d5f800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:48.407254 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-f8095716-cdd3-4f9e-b0da-298803d5f800 None None] MFA Rules not processed for user `a0d9868a89514f7590e7eb84915aa260`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:32:48.408468 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-f8095716-cdd3-4f9e-b0da-298803d5f800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:32:48.531581 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 829/1662] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:32:48 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:32:48.536729 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39fb591-e636-4351-83b5-836d0a90b160 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:48.677463 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39fb591-e636-4351-83b5-836d0a90b160 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:48.690491 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e9507f-7a32-4cf4-894d-582f5ea26957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:48.691639 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e9507f-7a32-4cf4-894d-582f5ea26957 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:48.691983 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e9507f-7a32-4cf4-894d-582f5ea26957 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:48.692263 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e9507f-7a32-4cf4-894d-582f5ea26957 admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:48.692948 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e9507f-7a32-4cf4-894d-582f5ea26957 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:48.695555 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e9507f-7a32-4cf4-894d-582f5ea26957 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:48.759008 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 834/1663] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Aug 6 01:32:48 2022] POST /identity/v3/projects => generated 407 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:32:48.760251 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8095716-cdd3-4f9e-b0da-298803d5f800 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:48.773703 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8095716-cdd3-4f9e-b0da-298803d5f800 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:48.893091 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:32:48.909195 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f555b00f-50fd-4be2-acc4-ac3dff594181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:48.910398 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f555b00f-50fd-4be2-acc4-ac3dff594181 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:48.911383 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f555b00f-50fd-4be2-acc4-ac3dff594181 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:48.911843 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f555b00f-50fd-4be2-acc4-ac3dff594181 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:48.912761 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f555b00f-50fd-4be2-acc4-ac3dff594181 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:48.914916 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f555b00f-50fd-4be2-acc4-ac3dff594181 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:48.967962 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 830/1664] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:32:48 2022] POST /identity/v3/users => generated 310 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:32:48.968829 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e9507f-7a32-4cf4-894d-582f5ea26957 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:48.992143 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e9507f-7a32-4cf4-894d-582f5ea26957 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:49.004633 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa10043-9875-4df6-925f-0f38a85dbbf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:49.005308 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa10043-9875-4df6-925f-0f38a85dbbf6 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:49.005591 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa10043-9875-4df6-925f-0f38a85dbbf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:49.005868 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa10043-9875-4df6-925f-0f38a85dbbf6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:49.006582 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa10043-9875-4df6-925f-0f38a85dbbf6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:49.008315 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa10043-9875-4df6-925f-0f38a85dbbf6 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:49.045559 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 835/1665] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:32:48 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 01:32:49.050858 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f555b00f-50fd-4be2-acc4-ac3dff594181 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:49.083220 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f555b00f-50fd-4be2-acc4-ac3dff594181 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:49.101278 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a5c24f-fdcb-47c7-bda0-f86c9d3f3954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:49.103679 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a5c24f-fdcb-47c7-bda0-f86c9d3f3954 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:49.105393 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a5c24f-fdcb-47c7-bda0-f86c9d3f3954 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:49.106420 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a5c24f-fdcb-47c7-bda0-f86c9d3f3954 admin admin] PATH_INFO: `/v3/projects/02d40069b7174b7d8968b390b2ee3581/users/d3c183fa7abc4980a56cf07e6d69c459/roles/4559b9a0837249f895a13c70096f8d38` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:49.154320 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a5c24f-fdcb-47c7-bda0-f86c9d3f3954 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02d40069b7174b7d8968b390b2ee3581, user_id=d3c183fa7abc4980a56cf07e6d69c459, role_id=4559b9a0837249f895a13c70096f8d38)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:49.166360 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a5c24f-fdcb-47c7-bda0-f86c9d3f3954 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:49.206910 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa10043-9875-4df6-925f-0f38a85dbbf6 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:49.210302 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-4aa10043-9875-4df6-925f-0f38a85dbbf6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:32:49.224845 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 831/1666] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:32:49 2022] PUT /identity/v3/projects/02d40069b7174b7d8968b390b2ee3581/users/d3c183fa7abc4980a56cf07e6d69c459/roles/4559b9a0837249f895a13c70096f8d38 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:32:49.227010 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a5c24f-fdcb-47c7-bda0-f86c9d3f3954 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:49.232606 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa10043-9875-4df6-925f-0f38a85dbbf6 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:49.354755 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:32:49.382755 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0b233d-bcc6-4f5d-a5c5-777cac5ab72a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:49.383424 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0b233d-bcc6-4f5d-a5c5-777cac5ab72a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:49.383705 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0b233d-bcc6-4f5d-a5c5-777cac5ab72a admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:49.384214 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0b233d-bcc6-4f5d-a5c5-777cac5ab72a admin admin] PATH_INFO: `/v3/projects/a63fdb77fdfa4bd5943a72ffd59c7ce9` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:49.388464 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0b233d-bcc6-4f5d-a5c5-777cac5ab72a admin admin] RBAC: Authorizing `identity:delete_project(project_id=a63fdb77fdfa4bd5943a72ffd59c7ce9)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:49.398231 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0b233d-bcc6-4f5d-a5c5-777cac5ab72a admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:49.479408 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a5c24f-fdcb-47c7-bda0-f86c9d3f3954 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:49.505943 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20546b29-95bb-4809-953d-7fe030c5239f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:49.506604 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20546b29-95bb-4809-953d-7fe030c5239f admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:49.506890 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20546b29-95bb-4809-953d-7fe030c5239f admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:49.507168 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20546b29-95bb-4809-953d-7fe030c5239f admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:49.512453 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20546b29-95bb-4809-953d-7fe030c5239f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:49.514123 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20546b29-95bb-4809-953d-7fe030c5239f admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:49.569657 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 832/1667] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:32:49 2022] GET /identity/v3/roles => generated 1420 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:32:49.571241 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20546b29-95bb-4809-953d-7fe030c5239f admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:49.600279 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-9e0b233d-bcc6-4f5d-a5c5-777cac5ab72a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a63fdb77fdfa4bd5943a72ffd59c7ce9', 'request_id': 'req-9e0b233d-bcc6-4f5d-a5c5-777cac5ab72a'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:32:49.620356 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20546b29-95bb-4809-953d-7fe030c5239f admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:49.622271 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-9e0b233d-bcc6-4f5d-a5c5-777cac5ab72a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a63fdb77fdfa4bd5943a72ffd59c7ce9', 'request_id': 'req-9e0b233d-bcc6-4f5d-a5c5-777cac5ab72a'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:32:49.654098 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-9e0b233d-bcc6-4f5d-a5c5-777cac5ab72a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a63fdb77fdfa4bd5943a72ffd59c7ce9', 'request_id': 'req-9e0b233d-bcc6-4f5d-a5c5-777cac5ab72a'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:32:49.702869 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-9e0b233d-bcc6-4f5d-a5c5-777cac5ab72a admin admin] The token cache is being invalidate because project a63fdb77fdfa4bd5943a72ffd59c7ce9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:32:49.703175 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-9e0b233d-bcc6-4f5d-a5c5-777cac5ab72a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:32:49.703855 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-9e0b233d-bcc6-4f5d-a5c5-777cac5ab72a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:32:49.708062 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 836/1668] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Aug 6 01:32:49 2022] DELETE /identity/v3/projects/a63fdb77fdfa4bd5943a72ffd59c7ce9 => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:32:49.720425 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553f0a35-3ad8-46df-bae1-4731abc1630e None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:49.720712 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553f0a35-3ad8-46df-bae1-4731abc1630e None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:49.720992 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553f0a35-3ad8-46df-bae1-4731abc1630e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:49.788948 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-553f0a35-3ad8-46df-bae1-4731abc1630e None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:32:49.795224 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-553f0a35-3ad8-46df-bae1-4731abc1630e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:32:49.865480 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f91ea29-0a68-422a-84ed-eaf6c4b95d0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:49.870861 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f91ea29-0a68-422a-84ed-eaf6c4b95d0b admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:49.872488 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f91ea29-0a68-422a-84ed-eaf6c4b95d0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:49.874380 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f91ea29-0a68-422a-84ed-eaf6c4b95d0b admin admin] PATH_INFO: `/v3/projects/02d40069b7174b7d8968b390b2ee3581/users/d3c183fa7abc4980a56cf07e6d69c459/roles/4559b9a0837249f895a13c70096f8d38` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:49.884856 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f91ea29-0a68-422a-84ed-eaf6c4b95d0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=02d40069b7174b7d8968b390b2ee3581, user_id=d3c183fa7abc4980a56cf07e6d69c459, role_id=4559b9a0837249f895a13c70096f8d38)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:49.894419 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f91ea29-0a68-422a-84ed-eaf6c4b95d0b admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:49.950306 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 833/1669] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:32:49 2022] PUT /identity/v3/projects/02d40069b7174b7d8968b390b2ee3581/users/d3c183fa7abc4980a56cf07e6d69c459/roles/4559b9a0837249f895a13c70096f8d38 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:32:49.964101 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7684754c-aa38-425b-bbdd-64abaa6e77e9 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:49.964464 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7684754c-aa38-425b-bbdd-64abaa6e77e9 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:49.964691 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7684754c-aa38-425b-bbdd-64abaa6e77e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:50.094304 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-7684754c-aa38-425b-bbdd-64abaa6e77e9 None None] MFA Rules not processed for user `d3c183fa7abc4980a56cf07e6d69c459`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:32:50.095902 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-7684754c-aa38-425b-bbdd-64abaa6e77e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:32:50.102123 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 837/1670] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:32:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:32:50.103646 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfc22af-a7a6-4d20-b6a9-ceac68c16da5 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:50.103941 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfc22af-a7a6-4d20-b6a9-ceac68c16da5 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:50.104228 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfc22af-a7a6-4d20-b6a9-ceac68c16da5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:50.163011 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-8cfc22af-a7a6-4d20-b6a9-ceac68c16da5 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:32:50.166384 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-8cfc22af-a7a6-4d20-b6a9-ceac68c16da5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:32:50.239066 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:32:50.338784 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 834/1671] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:32:49 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 376 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:32:50.349204 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7684754c-aa38-425b-bbdd-64abaa6e77e9 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:50.353358 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-7684754c-aa38-425b-bbdd-64abaa6e77e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:32:50.361391 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 838/1672] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:32:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:32:50.362962 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfc22af-a7a6-4d20-b6a9-ceac68c16da5 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:50.383260 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfc22af-a7a6-4d20-b6a9-ceac68c16da5 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:50.399119 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8ebc45-792b-448d-b524-433b6a054e18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:50.400306 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8ebc45-792b-448d-b524-433b6a054e18 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:50.400306 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8ebc45-792b-448d-b524-433b6a054e18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:50.402044 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8ebc45-792b-448d-b524-433b6a054e18 admin admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:50.402463 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8ebc45-792b-448d-b524-433b6a054e18 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:32:50.403011 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8ebc45-792b-448d-b524-433b6a054e18 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:50.404746 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8ebc45-792b-448d-b524-433b6a054e18 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:50.453772 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 839/1673] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Aug 6 01:32:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:32:50.458889 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8ebc45-792b-448d-b524-433b6a054e18 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:50.480702 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8ebc45-792b-448d-b524-433b6a054e18 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:50.497697 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e090ee-f64c-41c0-acf0-1a9e48439134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:50.498716 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e090ee-f64c-41c0-acf0-1a9e48439134 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:50.499205 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e090ee-f64c-41c0-acf0-1a9e48439134 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:50.499670 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e090ee-f64c-41c0-acf0-1a9e48439134 admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:50.500523 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e090ee-f64c-41c0-acf0-1a9e48439134 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:50.502867 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e090ee-f64c-41c0-acf0-1a9e48439134 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:50.520173 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:32:50.575864 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 840/1674] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Aug 6 01:32:50 2022] POST /identity/v3/projects => generated 373 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:32:50.576827 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e090ee-f64c-41c0-acf0-1a9e48439134 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:50.589520 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7684754c-aa38-425b-bbdd-64abaa6e77e9 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:50.766692 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160e8197-3096-4640-85a8-795c07d13098 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1be1e6173e374a60b9e1fbf51a1ab571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:50.767350 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160e8197-3096-4640-85a8-795c07d13098 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:50.767694 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160e8197-3096-4640-85a8-795c07d13098 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:50.768076 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160e8197-3096-4640-85a8-795c07d13098 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:50.775317 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e090ee-f64c-41c0-acf0-1a9e48439134 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:50.793975 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c21369-7a81-41e9-8072-a2f7c0476d2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:50.794633 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c21369-7a81-41e9-8072-a2f7c0476d2a admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:50.796131 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c21369-7a81-41e9-8072-a2f7c0476d2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:50.796411 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c21369-7a81-41e9-8072-a2f7c0476d2a admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:50.805218 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c21369-7a81-41e9-8072-a2f7c0476d2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:50.807216 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c21369-7a81-41e9-8072-a2f7c0476d2a admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:50.865333 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 841/1675] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:32:50 2022] POST /identity/v3/users => generated 287 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:32:50.870607 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c21369-7a81-41e9-8072-a2f7c0476d2a admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:50.899209 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c21369-7a81-41e9-8072-a2f7c0476d2a admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:50.910681 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d5ca7a-186e-4b10-95a2-5771ae1e67dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:50.911334 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d5ca7a-186e-4b10-95a2-5771ae1e67dd admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:50.911632 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d5ca7a-186e-4b10-95a2-5771ae1e67dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:50.911883 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d5ca7a-186e-4b10-95a2-5771ae1e67dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:50.912791 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d5ca7a-186e-4b10-95a2-5771ae1e67dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:50.934861 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d5ca7a-186e-4b10-95a2-5771ae1e67dd admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:50.966581 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160e8197-3096-4640-85a8-795c07d13098 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:50.968288 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160e8197-3096-4640-85a8-795c07d13098 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:51.001658 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 842/1676] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:32:50 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:32:51.006065 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d5ca7a-186e-4b10-95a2-5771ae1e67dd admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:51.032887 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d5ca7a-186e-4b10-95a2-5771ae1e67dd admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:51.036779 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 835/1677] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:32:50 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 689 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:32:51.039265 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d1ac48-ebfb-45e9-9331-447f5e88c76c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:51.040407 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d1ac48-ebfb-45e9-9331-447f5e88c76c admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:51.040895 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d1ac48-ebfb-45e9-9331-447f5e88c76c admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:51.041415 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d1ac48-ebfb-45e9-9331-447f5e88c76c admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:51.042542 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d1ac48-ebfb-45e9-9331-447f5e88c76c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:51.044620 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d1ac48-ebfb-45e9-9331-447f5e88c76c admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:51.059252 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 843/1678] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:32:51 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:32:51.064435 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160e8197-3096-4640-85a8-795c07d13098 service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:51.107431 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160e8197-3096-4640-85a8-795c07d13098 service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:51.149839 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e767ac-f75e-4101-833d-81bb9ecaae4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:51.151059 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e767ac-f75e-4101-833d-81bb9ecaae4e admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:51.151711 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e767ac-f75e-4101-833d-81bb9ecaae4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:51.151711 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e767ac-f75e-4101-833d-81bb9ecaae4e admin admin] PATH_INFO: `/v3/projects/d183e0a91de84b02ae7cd941c95a1040/users/3ebae341795649adba068af7f95c3992/roles/61b3b93d37a047e4bb8369b46fe20db9` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:51.214119 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e767ac-f75e-4101-833d-81bb9ecaae4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d183e0a91de84b02ae7cd941c95a1040, user_id=3ebae341795649adba068af7f95c3992, role_id=61b3b93d37a047e4bb8369b46fe20db9)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:51.218638 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e767ac-f75e-4101-833d-81bb9ecaae4e admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:51.260737 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 836/1679] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:32:51 2022] PUT /identity/v3/projects/d183e0a91de84b02ae7cd941c95a1040/users/3ebae341795649adba068af7f95c3992/roles/61b3b93d37a047e4bb8369b46fe20db9 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:32:51.273007 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32677a47-8f8c-433f-95cb-6e2d79cc6b5f None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:51.273555 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32677a47-8f8c-433f-95cb-6e2d79cc6b5f None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:51.273555 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32677a47-8f8c-433f-95cb-6e2d79cc6b5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:51.374979 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-32677a47-8f8c-433f-95cb-6e2d79cc6b5f None None] MFA Rules not processed for user `3ebae341795649adba068af7f95c3992`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:32:51.376155 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-32677a47-8f8c-433f-95cb-6e2d79cc6b5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:32:51.571085 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e767ac-f75e-4101-833d-81bb9ecaae4e admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:51.573313 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-32e767ac-f75e-4101-833d-81bb9ecaae4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:32:51.606582 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:32:51.607774 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 844/1680] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:32:51 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 336 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:32:51.611900 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32677a47-8f8c-433f-95cb-6e2d79cc6b5f None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:51.711242 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e767ac-f75e-4101-833d-81bb9ecaae4e admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:51.717366 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61424c25-9d0b-4789-aed4-41d1a919c4bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1be1e6173e374a60b9e1fbf51a1ab571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:51.718282 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61424c25-9d0b-4789-aed4-41d1a919c4bd service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:51.718771 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61424c25-9d0b-4789-aed4-41d1a919c4bd service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:51.719243 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61424c25-9d0b-4789-aed4-41d1a919c4bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:51.721414 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-61424c25-9d0b-4789-aed4-41d1a919c4bd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:32:51.726023 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:32:51.750559 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32677a47-8f8c-433f-95cb-6e2d79cc6b5f None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:51.757223 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6b39a0-b00a-4532-9b47-d4614b041f68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:51.758048 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6b39a0-b00a-4532-9b47-d4614b041f68 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:51.758553 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6b39a0-b00a-4532-9b47-d4614b041f68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:51.758933 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6b39a0-b00a-4532-9b47-d4614b041f68 admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:51.759712 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6b39a0-b00a-4532-9b47-d4614b041f68 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:51.763551 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6b39a0-b00a-4532-9b47-d4614b041f68 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:51.803589 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 845/1681] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Aug 6 01:32:51 2022] POST /identity/v3/projects => generated 373 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:32:51.808481 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6b39a0-b00a-4532-9b47-d4614b041f68 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:51.828534 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61424c25-9d0b-4789-aed4-41d1a919c4bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:51.865642 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61424c25-9d0b-4789-aed4-41d1a919c4bd service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:51.914275 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 837/1682] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:32:51 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 01:32:51.916252 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40232d11-477d-4fec-bc10-c4333845137d None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:51.916550 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40232d11-477d-4fec-bc10-c4333845137d None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:51.929489 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40232d11-477d-4fec-bc10-c4333845137d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:51.942039 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6b39a0-b00a-4532-9b47-d4614b041f68 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:51.953635 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f05afa-f5c3-4a71-a857-1512bce4d2bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:51.954280 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f05afa-f5c3-4a71-a857-1512bce4d2bd admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:51.954567 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f05afa-f5c3-4a71-a857-1512bce4d2bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:51.957702 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f05afa-f5c3-4a71-a857-1512bce4d2bd admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:51.960796 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f05afa-f5c3-4a71-a857-1512bce4d2bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:51.964762 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f05afa-f5c3-4a71-a857-1512bce4d2bd admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:51.989887 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-40232d11-477d-4fec-bc10-c4333845137d None None] MFA Rules not processed for user `5823f66f313843ddac68ff7a951321ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:32:51.993555 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-40232d11-477d-4fec-bc10-c4333845137d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:32:52.018732 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 846/1683] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:32:51 2022] POST /identity/v3/users => generated 293 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:32:52.021029 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f05afa-f5c3-4a71-a857-1512bce4d2bd admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:52.044871 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f05afa-f5c3-4a71-a857-1512bce4d2bd admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:52.058222 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaf9d41-cff6-49c1-9b82-e9f1da123f59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:52.058881 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaf9d41-cff6-49c1-9b82-e9f1da123f59 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:52.059163 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaf9d41-cff6-49c1-9b82-e9f1da123f59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:52.059441 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaf9d41-cff6-49c1-9b82-e9f1da123f59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:52.060409 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaf9d41-cff6-49c1-9b82-e9f1da123f59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:52.069333 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaf9d41-cff6-49c1-9b82-e9f1da123f59 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:52.096283 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 847/1684] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:32:52 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:32:52.099319 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaf9d41-cff6-49c1-9b82-e9f1da123f59 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:52.118695 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaf9d41-cff6-49c1-9b82-e9f1da123f59 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:52.134898 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe28913-8d1c-4937-a956-4148243fba7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:52.137426 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe28913-8d1c-4937-a956-4148243fba7b admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:52.137629 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe28913-8d1c-4937-a956-4148243fba7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:52.137970 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe28913-8d1c-4937-a956-4148243fba7b admin admin] PATH_INFO: `/v3/projects/01bf7a59ecc645b5ab2268f4d2215525/users/560ae6277a21472a873c633f38374d5f/roles/4559b9a0837249f895a13c70096f8d38` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:52.147456 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 838/1685] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:32:51 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:32:52.162621 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a146cb-bf19-459d-9cbc-ecdfb466c396 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:52.163179 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a146cb-bf19-459d-9cbc-ecdfb466c396 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:52.163747 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a146cb-bf19-459d-9cbc-ecdfb466c396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:52.166280 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe28913-8d1c-4937-a956-4148243fba7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=01bf7a59ecc645b5ab2268f4d2215525, user_id=560ae6277a21472a873c633f38374d5f, role_id=4559b9a0837249f895a13c70096f8d38)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:52.169095 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe28913-8d1c-4937-a956-4148243fba7b admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:52.194881 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 848/1686] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:32:52 2022] PUT /identity/v3/projects/01bf7a59ecc645b5ab2268f4d2215525/users/560ae6277a21472a873c633f38374d5f/roles/4559b9a0837249f895a13c70096f8d38 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:32:52.196356 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-62a146cb-bf19-459d-9cbc-ecdfb466c396 None None] MFA Rules not processed for user `5823f66f313843ddac68ff7a951321ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:32:52.198000 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-62a146cb-bf19-459d-9cbc-ecdfb466c396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:32:52.199836 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe28913-8d1c-4937-a956-4148243fba7b admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:52.341637 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 839/1687] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:32:52 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:32:52.370415 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a146cb-bf19-459d-9cbc-ecdfb466c396 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:52.371936 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-62a146cb-bf19-459d-9cbc-ecdfb466c396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:32:52.380357 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:32:52.399519 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe28913-8d1c-4937-a956-4148243fba7b admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:52.425623 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c7c89b-2989-40f2-9ba2-9d36a01b0b03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:52.430521 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c7c89b-2989-40f2-9ba2-9d36a01b0b03 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:52.430704 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c7c89b-2989-40f2-9ba2-9d36a01b0b03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:52.430978 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c7c89b-2989-40f2-9ba2-9d36a01b0b03 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:52.431875 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c7c89b-2989-40f2-9ba2-9d36a01b0b03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:52.435864 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c7c89b-2989-40f2-9ba2-9d36a01b0b03 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:52.463853 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 849/1688] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:32:52 2022] GET /identity/v3/roles => generated 1420 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:32:52.468061 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c7c89b-2989-40f2-9ba2-9d36a01b0b03 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:52.494075 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c7c89b-2989-40f2-9ba2-9d36a01b0b03 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:52.512758 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f227d380-74de-4c7d-94b9-075c28add0f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:52.513737 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f227d380-74de-4c7d-94b9-075c28add0f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:52.513899 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f227d380-74de-4c7d-94b9-075c28add0f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:52.514198 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f227d380-74de-4c7d-94b9-075c28add0f7 admin admin] PATH_INFO: `/v3/projects/01bf7a59ecc645b5ab2268f4d2215525/users/560ae6277a21472a873c633f38374d5f/roles/4559b9a0837249f895a13c70096f8d38` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:52.519195 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f227d380-74de-4c7d-94b9-075c28add0f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01bf7a59ecc645b5ab2268f4d2215525, user_id=560ae6277a21472a873c633f38374d5f, role_id=4559b9a0837249f895a13c70096f8d38)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:52.525226 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f227d380-74de-4c7d-94b9-075c28add0f7 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:52.574425 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 850/1689] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:32:52 2022] PUT /identity/v3/projects/01bf7a59ecc645b5ab2268f4d2215525/users/560ae6277a21472a873c633f38374d5f/roles/4559b9a0837249f895a13c70096f8d38 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:32:52.576289 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551cd16a-36a3-438c-8775-96a3abfd2327 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:52.576594 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551cd16a-36a3-438c-8775-96a3abfd2327 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:52.576942 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551cd16a-36a3-438c-8775-96a3abfd2327 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:52.591579 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a146cb-bf19-459d-9cbc-ecdfb466c396 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:52.608120 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:32:52.643773 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-551cd16a-36a3-438c-8775-96a3abfd2327 None None] MFA Rules not processed for user `560ae6277a21472a873c633f38374d5f`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:32:52.648800 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-551cd16a-36a3-438c-8775-96a3abfd2327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:32:52.749574 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8535dd2-0243-4104-a7a8-01e9524e1f74 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87adc1d5bc240e99f95f6595b350be4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:52.750946 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8535dd2-0243-4104-a7a8-01e9524e1f74 service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:52.751310 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8535dd2-0243-4104-a7a8-01e9524e1f74 service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:52.751663 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8535dd2-0243-4104-a7a8-01e9524e1f74 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:52.851004 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 851/1690] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:32:52 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:32:52.867734 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551cd16a-36a3-438c-8775-96a3abfd2327 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:52.908354 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8535dd2-0243-4104-a7a8-01e9524e1f74 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:52.910880 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8535dd2-0243-4104-a7a8-01e9524e1f74 service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:52.927727 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 840/1691] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:32:52 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 557 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:32:52.961415 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551cd16a-36a3-438c-8775-96a3abfd2327 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:52.965578 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0289a3-2881-4b71-85ce-c7e7b7f6d651 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1be1e6173e374a60b9e1fbf51a1ab571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:52.966542 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0289a3-2881-4b71-85ce-c7e7b7f6d651 service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:52.966821 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0289a3-2881-4b71-85ce-c7e7b7f6d651 service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:52.967095 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0289a3-2881-4b71-85ce-c7e7b7f6d651 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:52.978441 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0289a3-2881-4b71-85ce-c7e7b7f6d651 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:52.979893 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0289a3-2881-4b71-85ce-c7e7b7f6d651 service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:52.994960 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 852/1692] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:32:52 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 01:32:53.139101 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8535dd2-0243-4104-a7a8-01e9524e1f74 service nova] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:53.152290 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8535dd2-0243-4104-a7a8-01e9524e1f74 service nova] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:53.156466 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c73f282-e33e-4ed4-8851-e4cef387178c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1be1e6173e374a60b9e1fbf51a1ab571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:53.157167 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c73f282-e33e-4ed4-8851-e4cef387178c service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:53.157456 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c73f282-e33e-4ed4-8851-e4cef387178c service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:53.157734 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c73f282-e33e-4ed4-8851-e4cef387178c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:53.219817 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:32:53.279020 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c73f282-e33e-4ed4-8851-e4cef387178c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:53.280992 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c73f282-e33e-4ed4-8851-e4cef387178c service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:53.293303 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 841/1693] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:32:53 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:32:53.465392 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0224e25-3952-429b-be5f-3c7f1c4e5f6d None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:53.465895 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0224e25-3952-429b-be5f-3c7f1c4e5f6d None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:53.465976 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0224e25-3952-429b-be5f-3c7f1c4e5f6d None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:53.467366 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 853/1694] 10.222.0.26 () {54 vars in 912 bytes} [Sat Aug 6 01:32:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:32:53.474854 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd46eada-98c1-4e3d-be1e-229e0f7ff94f None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:53.475552 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd46eada-98c1-4e3d-be1e-229e0f7ff94f None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:53.476040 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd46eada-98c1-4e3d-be1e-229e0f7ff94f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:53.609865 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-bd46eada-98c1-4e3d-be1e-229e0f7ff94f None None] MFA Rules not processed for user `4ac0e31c84b4445fa167bbe4954066c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:32:53.616155 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-bd46eada-98c1-4e3d-be1e-229e0f7ff94f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:32:53.696082 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 842/1695] 10.222.0.26 () {58 vars in 1009 bytes} [Sat Aug 6 01:32:53 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:32:53.759104 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0224e25-3952-429b-be5f-3c7f1c4e5f6d None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:53.760901 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-a0224e25-3952-429b-be5f-3c7f1c4e5f6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:32:53.786933 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0224e25-3952-429b-be5f-3c7f1c4e5f6d None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:53.797925 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73086661-7d97-40aa-8c8b-6b347b0015a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:53.798579 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73086661-7d97-40aa-8c8b-6b347b0015a6 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:53.798863 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73086661-7d97-40aa-8c8b-6b347b0015a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:53.799142 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73086661-7d97-40aa-8c8b-6b347b0015a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:53.815741 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73086661-7d97-40aa-8c8b-6b347b0015a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:53.817422 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73086661-7d97-40aa-8c8b-6b347b0015a6 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:53.835652 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 854/1696] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:32:53 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:32:54.803120 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd46eada-98c1-4e3d-be1e-229e0f7ff94f None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:54.808247 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-bd46eada-98c1-4e3d-be1e-229e0f7ff94f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:32:54.945874 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd46eada-98c1-4e3d-be1e-229e0f7ff94f None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:54.954533 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a185fa4a-16cd-4e3e-ac31-9edcf0bf7f51 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa3b93ddd9464a2cba724d67b56b477a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:54.955991 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a185fa4a-16cd-4e3e-ac31-9edcf0bf7f51 service placement] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:54.956341 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a185fa4a-16cd-4e3e-ac31-9edcf0bf7f51 service placement] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:54.957256 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a185fa4a-16cd-4e3e-ac31-9edcf0bf7f51 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:54.960071 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-a185fa4a-16cd-4e3e-ac31-9edcf0bf7f51 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:32:55.001761 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a185fa4a-16cd-4e3e-ac31-9edcf0bf7f51 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:55.004436 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a185fa4a-16cd-4e3e-ac31-9edcf0bf7f51 service placement] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:55.038460 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 843/1697] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:32:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:32:57.525631 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f83b4a-3967-4c06-a775-c01c63757a3d None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:57.526615 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f83b4a-3967-4c06-a775-c01c63757a3d None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:57.527042 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f83b4a-3967-4c06-a775-c01c63757a3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:57.598966 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-d8f83b4a-3967-4c06-a775-c01c63757a3d None None] MFA Rules not processed for user `2455ea8abba547669fe22d2065a726c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:32:57.600414 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-d8f83b4a-3967-4c06-a775-c01c63757a3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:32:57.804885 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 855/1698] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:32:57 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:32:57.817347 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a185fa4a-16cd-4e3e-ac31-9edcf0bf7f51 service placement] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:57.835200 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a185fa4a-16cd-4e3e-ac31-9edcf0bf7f51 service placement] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:57.877895 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64103ae-31a4-47d8-b9d5-93454637e25c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87adc1d5bc240e99f95f6595b350be4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:57.878718 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64103ae-31a4-47d8-b9d5-93454637e25c service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:57.879015 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64103ae-31a4-47d8-b9d5-93454637e25c service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:57.879367 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64103ae-31a4-47d8-b9d5-93454637e25c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:57.900540 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64103ae-31a4-47d8-b9d5-93454637e25c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:57.902392 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64103ae-31a4-47d8-b9d5-93454637e25c service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:57.915843 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 844/1699] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:32:57 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 01:32:59.986542 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b5a648-80ad-4800-8fd8-9d2a2d7822c7 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:59.987129 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b5a648-80ad-4800-8fd8-9d2a2d7822c7 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:59.988011 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b5a648-80ad-4800-8fd8-9d2a2d7822c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:00.075176 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:33:00.158888 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-a9b5a648-80ad-4800-8fd8-9d2a2d7822c7 None None] MFA Rules not processed for user `8e86f38590824894a3b960e6774ff8a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:33:00.161301 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-a9b5a648-80ad-4800-8fd8-9d2a2d7822c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:00.314583 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f7e172-ff2b-4941-ace7-c1605d9c0191 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:00.316791 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 856/1700] 10.222.0.26 () {58 vars in 1009 bytes} [Sat Aug 6 01:32:59 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 330 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:33:00.324618 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f7e172-ff2b-4941-ace7-c1605d9c0191 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:00.325545 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f7e172-ff2b-4941-ace7-c1605d9c0191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:00.429184 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-25f7e172-ff2b-4941-ace7-c1605d9c0191 None None] MFA Rules not processed for user `37c704ce557c4f309cf00ef32646235f`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:33:00.434007 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-25f7e172-ff2b-4941-ace7-c1605d9c0191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:00.524553 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b5a648-80ad-4800-8fd8-9d2a2d7822c7 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:00.526147 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-a9b5a648-80ad-4800-8fd8-9d2a2d7822c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:00.559246 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:33:00.635366 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 845/1701] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:33:00 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 322 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:33:00.652676 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0d31b5-87c6-4a40-b09e-9340b49e0671 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:00.653339 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0d31b5-87c6-4a40-b09e-9340b49e0671 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:00.654193 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0d31b5-87c6-4a40-b09e-9340b49e0671 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:00.684587 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-ab0d31b5-87c6-4a40-b09e-9340b49e0671 None None] MFA Rules not processed for user `37c704ce557c4f309cf00ef32646235f`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:33:00.686298 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-ab0d31b5-87c6-4a40-b09e-9340b49e0671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:00.699384 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 846/1702] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:33:00 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:33:00.724185 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0d31b5-87c6-4a40-b09e-9340b49e0671 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:00.725999 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-ab0d31b5-87c6-4a40-b09e-9340b49e0671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:00.770658 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0d31b5-87c6-4a40-b09e-9340b49e0671 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:00.791391 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f2be03-35a3-4ae2-99e7-aa7e84014b44 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87adc1d5bc240e99f95f6595b350be4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:00.792195 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f2be03-35a3-4ae2-99e7-aa7e84014b44 service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:00.792487 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f2be03-35a3-4ae2-99e7-aa7e84014b44 service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:00.792768 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f2be03-35a3-4ae2-99e7-aa7e84014b44 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:00.794841 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b5a648-80ad-4800-8fd8-9d2a2d7822c7 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:00.802355 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c38a24-6033-4c4a-b00c-01d6e78a665d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b23f4478f49471fba6f43e6e4be99da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:00.803285 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c38a24-6033-4c4a-b00c-01d6e78a665d service swift] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:00.804240 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c38a24-6033-4c4a-b00c-01d6e78a665d service swift] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:00.804721 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c38a24-6033-4c4a-b00c-01d6e78a665d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:00.814430 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f2be03-35a3-4ae2-99e7-aa7e84014b44 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:00.816975 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f2be03-35a3-4ae2-99e7-aa7e84014b44 service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:00.830349 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c38a24-6033-4c4a-b00c-01d6e78a665d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:00.833360 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c38a24-6033-4c4a-b00c-01d6e78a665d service swift] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:00.840464 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 847/1703] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:33:00 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 01:33:00.842260 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1606f5-4b45-4448-ba6f-7acc97b8e513 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:00.842746 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1606f5-4b45-4448-ba6f-7acc97b8e513 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:00.843961 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1606f5-4b45-4448-ba6f-7acc97b8e513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:00.871341 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-cb1606f5-4b45-4448-ba6f-7acc97b8e513 None None] MFA Rules not processed for user `4ac0e31c84b4445fa167bbe4954066c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:33:00.872867 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-cb1606f5-4b45-4448-ba6f-7acc97b8e513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:00.888922 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 848/1704] 10.222.0.33 () {58 vars in 1015 bytes} [Sat Aug 6 01:33:00 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:33:00.892030 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 857/1705] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Aug 6 01:33:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 06 01:33:01.812360 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c8faa5-055c-49b6-a758-0743029b4b68 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:01.813635 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c8faa5-055c-49b6-a758-0743029b4b68 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:01.814271 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c8faa5-055c-49b6-a758-0743029b4b68 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:01.825185 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 849/1706] 10.222.0.26 () {54 vars in 912 bytes} [Sat Aug 6 01:33:01 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:33:01.828806 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3817a3be-78d5-45fe-8950-78438df3f232 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:01.829362 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3817a3be-78d5-45fe-8950-78438df3f232 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:01.829362 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3817a3be-78d5-45fe-8950-78438df3f232 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:01.855430 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-3817a3be-78d5-45fe-8950-78438df3f232 None None] MFA Rules not processed for user `4ac0e31c84b4445fa167bbe4954066c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:33:01.857072 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-3817a3be-78d5-45fe-8950-78438df3f232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:01.876436 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 858/1707] 10.222.0.26 () {58 vars in 1009 bytes} [Sat Aug 6 01:33:01 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:33:02.023353 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c8faa5-055c-49b6-a758-0743029b4b68 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:02.043722 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c8faa5-055c-49b6-a758-0743029b4b68 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:02.056741 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f883d8-a347-4d96-a5dc-8b29c9ded8ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:02.058592 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f883d8-a347-4d96-a5dc-8b29c9ded8ba service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:02.058874 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f883d8-a347-4d96-a5dc-8b29c9ded8ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:02.062422 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f883d8-a347-4d96-a5dc-8b29c9ded8ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:02.085876 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f883d8-a347-4d96-a5dc-8b29c9ded8ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:02.087606 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f883d8-a347-4d96-a5dc-8b29c9ded8ba service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:02.107333 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 850/1708] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:33:02 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:33:02.328870 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ba1d59-d4a3-401e-8286-8f62095b6cf1 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:02.329454 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ba1d59-d4a3-401e-8286-8f62095b6cf1 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:02.329739 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ba1d59-d4a3-401e-8286-8f62095b6cf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:02.403220 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-06ba1d59-d4a3-401e-8286-8f62095b6cf1 None None] MFA Rules not processed for user `42ade772e11a4725928ab12d3bccc4a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:33:02.404458 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-06ba1d59-d4a3-401e-8286-8f62095b6cf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:02.489101 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:33:02.552788 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 859/1709] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:33:02 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:33:02.569946 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f883d8-a347-4d96-a5dc-8b29c9ded8ba service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:02.606644 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f883d8-a347-4d96-a5dc-8b29c9ded8ba service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:02.620888 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a917e3ef-602b-4e7d-b78c-e1045f2a2ee0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87adc1d5bc240e99f95f6595b350be4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:02.621581 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a917e3ef-602b-4e7d-b78c-e1045f2a2ee0 service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:02.621872 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a917e3ef-602b-4e7d-b78c-e1045f2a2ee0 service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:02.622152 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a917e3ef-602b-4e7d-b78c-e1045f2a2ee0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:02.643304 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a917e3ef-602b-4e7d-b78c-e1045f2a2ee0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:02.647034 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a917e3ef-602b-4e7d-b78c-e1045f2a2ee0 service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:02.660032 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 851/1710] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:33:02 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:33:03.192854 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbda3914-aecf-4543-826e-ccc3fe40b48f None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:03.193999 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbda3914-aecf-4543-826e-ccc3fe40b48f None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:03.196681 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfe17ea-5155-4dc0-8ed5-1b4b9d9f84e4 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:03.197177 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfe17ea-5155-4dc0-8ed5-1b4b9d9f84e4 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:03.197605 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbda3914-aecf-4543-826e-ccc3fe40b48f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:03.197874 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfe17ea-5155-4dc0-8ed5-1b4b9d9f84e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:03.272003 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-cbda3914-aecf-4543-826e-ccc3fe40b48f None None] MFA Rules not processed for user `2455ea8abba547669fe22d2065a726c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:33:03.273305 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-cbda3914-aecf-4543-826e-ccc3fe40b48f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:03.287489 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-7bfe17ea-5155-4dc0-8ed5-1b4b9d9f84e4 None None] MFA Rules not processed for user `37c704ce557c4f309cf00ef32646235f`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:33:03.289652 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-7bfe17ea-5155-4dc0-8ed5-1b4b9d9f84e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:03.306740 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 860/1711] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:33:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:33:03.325783 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 852/1712] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:33:03 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:33:03.527384 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbda3914-aecf-4543-826e-ccc3fe40b48f None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:03.531816 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-cbda3914-aecf-4543-826e-ccc3fe40b48f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:03.570939 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbda3914-aecf-4543-826e-ccc3fe40b48f None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:03.603405 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f84b04-f098-4eb8-946f-e1f4eb0c8638 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:03.605939 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f84b04-f098-4eb8-946f-e1f4eb0c8638 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:03.610373 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f84b04-f098-4eb8-946f-e1f4eb0c8638 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:03.614092 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f84b04-f098-4eb8-946f-e1f4eb0c8638 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:03.646560 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f84b04-f098-4eb8-946f-e1f4eb0c8638 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:03.653235 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f84b04-f098-4eb8-946f-e1f4eb0c8638 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:03.714643 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 861/1713] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:33:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:33:05.387517 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf2c7a4-8e74-48b0-9ef0-12b5d7db8712 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:05.388077 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf2c7a4-8e74-48b0-9ef0-12b5d7db8712 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:05.388077 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf2c7a4-8e74-48b0-9ef0-12b5d7db8712 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:05.390129 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 853/1714] 10.222.0.26 () {54 vars in 921 bytes} [Sat Aug 6 01:33:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:33:05.400954 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98c7715-1d92-40dc-bba4-33e5686bfb13 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:05.401445 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98c7715-1d92-40dc-bba4-33e5686bfb13 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:05.401526 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98c7715-1d92-40dc-bba4-33e5686bfb13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:05.447614 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-e98c7715-1d92-40dc-bba4-33e5686bfb13 None None] MFA Rules not processed for user `b87adc1d5bc240e99f95f6595b350be4`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:33:05.452491 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-e98c7715-1d92-40dc-bba4-33e5686bfb13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:05.495015 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 862/1715] 10.222.0.26 () {58 vars in 1018 bytes} [Sat Aug 6 01:33:05 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:33:06.259367 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf2c7a4-8e74-48b0-9ef0-12b5d7db8712 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:06.290025 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf2c7a4-8e74-48b0-9ef0-12b5d7db8712 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:06.297858 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58a6052-01b7-4a77-a214-129262ba81a2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87adc1d5bc240e99f95f6595b350be4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:06.298729 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58a6052-01b7-4a77-a214-129262ba81a2 service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:06.299319 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58a6052-01b7-4a77-a214-129262ba81a2 service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:06.300067 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58a6052-01b7-4a77-a214-129262ba81a2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:06.321022 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58a6052-01b7-4a77-a214-129262ba81a2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:06.323492 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58a6052-01b7-4a77-a214-129262ba81a2 service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:06.349180 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 854/1716] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:33:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:33:07.287406 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba4189f-684e-48e9-9e58-ca6df2700d4f None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:07.288115 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba4189f-684e-48e9-9e58-ca6df2700d4f None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:07.288221 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba4189f-684e-48e9-9e58-ca6df2700d4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:07.321404 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-eba4189f-684e-48e9-9e58-ca6df2700d4f None None] MFA Rules not processed for user `42ade772e11a4725928ab12d3bccc4a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:33:07.325741 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-eba4189f-684e-48e9-9e58-ca6df2700d4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:07.354090 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 863/1717] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:33:07 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:33:09.182332 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58a6052-01b7-4a77-a214-129262ba81a2 service nova] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:09.185597 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-a58a6052-01b7-4a77-a214-129262ba81a2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:09.642846 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58a6052-01b7-4a77-a214-129262ba81a2 service nova] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:09.657805 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15908a66-23ad-4b43-8247-fc31a1f92f76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:09.658740 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15908a66-23ad-4b43-8247-fc31a1f92f76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:09.659295 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15908a66-23ad-4b43-8247-fc31a1f92f76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:09.659733 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15908a66-23ad-4b43-8247-fc31a1f92f76 admin admin] PATH_INFO: `/v3/users/d588a87dd3bc4a97b75f06491445a7a8` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:09.664472 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15908a66-23ad-4b43-8247-fc31a1f92f76 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d588a87dd3bc4a97b75f06491445a7a8)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:09.670428 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15908a66-23ad-4b43-8247-fc31a1f92f76 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:09.804212 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-15908a66-23ad-4b43-8247-fc31a1f92f76 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd588a87dd3bc4a97b75f06491445a7a8'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:09.805934 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-15908a66-23ad-4b43-8247-fc31a1f92f76 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd588a87dd3bc4a97b75f06491445a7a8'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:09.808843 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-15908a66-23ad-4b43-8247-fc31a1f92f76 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd588a87dd3bc4a97b75f06491445a7a8'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:09.841148 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-15908a66-23ad-4b43-8247-fc31a1f92f76 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd588a87dd3bc4a97b75f06491445a7a8'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:09.843484 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:33:09.861502 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-15908a66-23ad-4b43-8247-fc31a1f92f76 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd588a87dd3bc4a97b75f06491445a7a8'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:09.891200 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 855/1718] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Aug 6 01:33:09 2022] DELETE /identity/v3/users/d588a87dd3bc4a97b75f06491445a7a8 => generated 0 bytes in 709 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:33:09.896302 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841cafec-f161-40eb-a97d-5d7aace7a910 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:09.897304 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841cafec-f161-40eb-a97d-5d7aace7a910 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:09.898138 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841cafec-f161-40eb-a97d-5d7aace7a910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:09.926139 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-841cafec-f161-40eb-a97d-5d7aace7a910 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:33:09.928194 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-841cafec-f161-40eb-a97d-5d7aace7a910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:10.071648 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 864/1719] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:33:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:33:10.087121 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15908a66-23ad-4b43-8247-fc31a1f92f76 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:10.090802 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-15908a66-23ad-4b43-8247-fc31a1f92f76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:10.168418 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841cafec-f161-40eb-a97d-5d7aace7a910 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:10.170356 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-841cafec-f161-40eb-a97d-5d7aace7a910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:10.196515 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841cafec-f161-40eb-a97d-5d7aace7a910 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:10.255979 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb2d791-d88a-4ff9-bc2d-3a14751462bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:10.258962 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb2d791-d88a-4ff9-bc2d-3a14751462bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:10.259450 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb2d791-d88a-4ff9-bc2d-3a14751462bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:10.259969 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb2d791-d88a-4ff9-bc2d-3a14751462bf admin admin] PATH_INFO: `/v3/users/a0d9868a89514f7590e7eb84915aa260` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:10.266063 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb2d791-d88a-4ff9-bc2d-3a14751462bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0d9868a89514f7590e7eb84915aa260)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:10.268946 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb2d791-d88a-4ff9-bc2d-3a14751462bf admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:10.269632 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15908a66-23ad-4b43-8247-fc31a1f92f76 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:10.281924 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836e59f2-f58f-4c86-8169-7bf62eab0971 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:10.282863 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836e59f2-f58f-4c86-8169-7bf62eab0971 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:10.285570 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836e59f2-f58f-4c86-8169-7bf62eab0971 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:10.286269 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836e59f2-f58f-4c86-8169-7bf62eab0971 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:10.312477 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836e59f2-f58f-4c86-8169-7bf62eab0971 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:10.314413 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836e59f2-f58f-4c86-8169-7bf62eab0971 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:10.344503 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 856/1720] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:33:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:33:10.468750 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-9fb2d791-d88a-4ff9-bc2d-3a14751462bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0d9868a89514f7590e7eb84915aa260'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:10.505726 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-9fb2d791-d88a-4ff9-bc2d-3a14751462bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0d9868a89514f7590e7eb84915aa260'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:10.509638 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-9fb2d791-d88a-4ff9-bc2d-3a14751462bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0d9868a89514f7590e7eb84915aa260'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:10.512728 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-9fb2d791-d88a-4ff9-bc2d-3a14751462bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0d9868a89514f7590e7eb84915aa260'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:10.578635 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-9fb2d791-d88a-4ff9-bc2d-3a14751462bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0d9868a89514f7590e7eb84915aa260'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:10.621298 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 865/1721] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Aug 6 01:33:10 2022] DELETE /identity/v3/users/a0d9868a89514f7590e7eb84915aa260 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:33:10.626099 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acc1615-dcd8-4cf6-8b0b-2c603d18d3ef None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:10.626963 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acc1615-dcd8-4cf6-8b0b-2c603d18d3ef None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:10.627633 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acc1615-dcd8-4cf6-8b0b-2c603d18d3ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:10.688522 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-2acc1615-dcd8-4cf6-8b0b-2c603d18d3ef None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:33:10.698703 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-2acc1615-dcd8-4cf6-8b0b-2c603d18d3ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:10.970045 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 857/1722] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:33:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 345 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:33:10.981032 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb2d791-d88a-4ff9-bc2d-3a14751462bf admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:10.983093 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-9fb2d791-d88a-4ff9-bc2d-3a14751462bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:11.071603 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:33:11.079029 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acc1615-dcd8-4cf6-8b0b-2c603d18d3ef None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:11.082191 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-2acc1615-dcd8-4cf6-8b0b-2c603d18d3ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:11.139892 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acc1615-dcd8-4cf6-8b0b-2c603d18d3ef None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:11.173533 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af51429-6d7a-4f22-a16a-e12ce48bcd33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:11.174184 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af51429-6d7a-4f22-a16a-e12ce48bcd33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:11.176127 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af51429-6d7a-4f22-a16a-e12ce48bcd33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:11.176432 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af51429-6d7a-4f22-a16a-e12ce48bcd33 admin admin] PATH_INFO: `/v3/users/fc49b67762b54904926edc482ccc6505` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:11.195792 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af51429-6d7a-4f22-a16a-e12ce48bcd33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc49b67762b54904926edc482ccc6505)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:11.211858 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af51429-6d7a-4f22-a16a-e12ce48bcd33 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:11.255561 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb2d791-d88a-4ff9-bc2d-3a14751462bf admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:11.271961 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0052a39c-de57-458d-afe3-d884ba627d79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:11.272622 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0052a39c-de57-458d-afe3-d884ba627d79 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:11.272912 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0052a39c-de57-458d-afe3-d884ba627d79 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:11.273221 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0052a39c-de57-458d-afe3-d884ba627d79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:11.313686 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0052a39c-de57-458d-afe3-d884ba627d79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:11.325074 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0052a39c-de57-458d-afe3-d884ba627d79 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:11.409989 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-6af51429-6d7a-4f22-a16a-e12ce48bcd33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc49b67762b54904926edc482ccc6505'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:11.415762 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-6af51429-6d7a-4f22-a16a-e12ce48bcd33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc49b67762b54904926edc482ccc6505'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:11.416983 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-6af51429-6d7a-4f22-a16a-e12ce48bcd33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc49b67762b54904926edc482ccc6505'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:11.417816 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 866/1723] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:33:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:33:11.451914 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-6af51429-6d7a-4f22-a16a-e12ce48bcd33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc49b67762b54904926edc482ccc6505'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:11.477429 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-6af51429-6d7a-4f22-a16a-e12ce48bcd33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc49b67762b54904926edc482ccc6505'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:11.512611 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:33:11.525079 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 858/1724] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Aug 6 01:33:11 2022] DELETE /identity/v3/users/fc49b67762b54904926edc482ccc6505 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:33:11.529417 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fb65f2-5ef2-4926-8b82-0e146b3eb93e None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:11.529866 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fb65f2-5ef2-4926-8b82-0e146b3eb93e None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:11.530154 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fb65f2-5ef2-4926-8b82-0e146b3eb93e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:11.604787 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-f5fb65f2-5ef2-4926-8b82-0e146b3eb93e None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:33:11.606044 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-f5fb65f2-5ef2-4926-8b82-0e146b3eb93e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:11.816395 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 867/1725] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:33:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:33:11.896836 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af51429-6d7a-4f22-a16a-e12ce48bcd33 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:11.898509 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-6af51429-6d7a-4f22-a16a-e12ce48bcd33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:12.083858 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af51429-6d7a-4f22-a16a-e12ce48bcd33 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:12.103947 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c8b448-6e61-4a9d-8448-89345e9b56a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:12.105030 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c8b448-6e61-4a9d-8448-89345e9b56a5 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:12.105754 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c8b448-6e61-4a9d-8448-89345e9b56a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:12.106189 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c8b448-6e61-4a9d-8448-89345e9b56a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:12.126017 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c8b448-6e61-4a9d-8448-89345e9b56a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:12.137134 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c8b448-6e61-4a9d-8448-89345e9b56a5 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:12.158507 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fb65f2-5ef2-4926-8b82-0e146b3eb93e None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:12.160506 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-f5fb65f2-5ef2-4926-8b82-0e146b3eb93e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:12.166864 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 859/1726] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:33:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:33:12.182842 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fb65f2-5ef2-4926-8b82-0e146b3eb93e None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:12.198337 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac875d2-3b27-468a-92e3-9d0bc801438d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:12.199060 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac875d2-3b27-468a-92e3-9d0bc801438d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:12.199336 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac875d2-3b27-468a-92e3-9d0bc801438d admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:12.199786 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac875d2-3b27-468a-92e3-9d0bc801438d admin admin] PATH_INFO: `/v3/projects/a778a936bf3c45d39e51c562209dfb64` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:12.202392 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac875d2-3b27-468a-92e3-9d0bc801438d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a778a936bf3c45d39e51c562209dfb64)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:12.207218 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac875d2-3b27-468a-92e3-9d0bc801438d admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:12.321701 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-bac875d2-3b27-468a-92e3-9d0bc801438d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a778a936bf3c45d39e51c562209dfb64', 'request_id': 'req-bac875d2-3b27-468a-92e3-9d0bc801438d'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:12.332151 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-bac875d2-3b27-468a-92e3-9d0bc801438d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a778a936bf3c45d39e51c562209dfb64', 'request_id': 'req-bac875d2-3b27-468a-92e3-9d0bc801438d'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:12.348144 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-bac875d2-3b27-468a-92e3-9d0bc801438d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a778a936bf3c45d39e51c562209dfb64', 'request_id': 'req-bac875d2-3b27-468a-92e3-9d0bc801438d'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:12.378215 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-bac875d2-3b27-468a-92e3-9d0bc801438d admin admin] The token cache is being invalidate because project a778a936bf3c45d39e51c562209dfb64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:33:12.378741 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-bac875d2-3b27-468a-92e3-9d0bc801438d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:12.379780 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-bac875d2-3b27-468a-92e3-9d0bc801438d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:12.382019 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 868/1727] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Aug 6 01:33:12 2022] DELETE /identity/v3/projects/a778a936bf3c45d39e51c562209dfb64 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:33:12.387225 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915d07c9-3e89-44c4-b636-78844e4aa14a None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:12.387587 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915d07c9-3e89-44c4-b636-78844e4aa14a None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:12.387953 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915d07c9-3e89-44c4-b636-78844e4aa14a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:12.447208 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-915d07c9-3e89-44c4-b636-78844e4aa14a None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:33:12.449082 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-915d07c9-3e89-44c4-b636-78844e4aa14a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:12.578386 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 860/1728] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:33:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:33:12.588385 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8abd7c-1c6e-4348-bde6-6df3896ef124 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:12.589202 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8abd7c-1c6e-4348-bde6-6df3896ef124 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:12.592051 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8abd7c-1c6e-4348-bde6-6df3896ef124 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:12.661862 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-4e8abd7c-1c6e-4348-bde6-6df3896ef124 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:33:12.666407 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-4e8abd7c-1c6e-4348-bde6-6df3896ef124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:12.699008 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 869/1729] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:33:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:33:12.703793 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915d07c9-3e89-44c4-b636-78844e4aa14a None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:12.721002 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915d07c9-3e89-44c4-b636-78844e4aa14a None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:12.739814 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8abd7c-1c6e-4348-bde6-6df3896ef124 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:12.740860 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03741eb6-0eeb-4f8b-b357-d7e0b2ca2da6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:12.741337 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-4e8abd7c-1c6e-4348-bde6-6df3896ef124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:12.741594 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03741eb6-0eeb-4f8b-b357-d7e0b2ca2da6 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:12.741878 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03741eb6-0eeb-4f8b-b357-d7e0b2ca2da6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:12.742156 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03741eb6-0eeb-4f8b-b357-d7e0b2ca2da6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:12.742869 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03741eb6-0eeb-4f8b-b357-d7e0b2ca2da6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:33:12.746299 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03741eb6-0eeb-4f8b-b357-d7e0b2ca2da6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:12.748002 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03741eb6-0eeb-4f8b-b357-d7e0b2ca2da6 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:12.766387 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8abd7c-1c6e-4348-bde6-6df3896ef124 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:12.767459 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 861/1730] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Aug 6 01:33:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:33:12.771949 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03741eb6-0eeb-4f8b-b357-d7e0b2ca2da6 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:12.786363 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03741eb6-0eeb-4f8b-b357-d7e0b2ca2da6 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:12.791942 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31094904-e1a3-42e5-b0d2-c4f95142d608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:12.792616 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31094904-e1a3-42e5-b0d2-c4f95142d608 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:12.793221 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31094904-e1a3-42e5-b0d2-c4f95142d608 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:12.793221 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31094904-e1a3-42e5-b0d2-c4f95142d608 admin admin] PATH_INFO: `/v3/projects/28e80afa761e41989ae14fe250b3656c` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:12.794428 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31094904-e1a3-42e5-b0d2-c4f95142d608 admin admin] RBAC: Authorizing `identity:delete_project(project_id=28e80afa761e41989ae14fe250b3656c)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:12.794562 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da1addf-e3bc-4c0b-bf37-8c9767a55a9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:12.795687 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da1addf-e3bc-4c0b-bf37-8c9767a55a9a admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:12.796022 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da1addf-e3bc-4c0b-bf37-8c9767a55a9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:12.796352 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da1addf-e3bc-4c0b-bf37-8c9767a55a9a admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:12.796463 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31094904-e1a3-42e5-b0d2-c4f95142d608 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:12.797029 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da1addf-e3bc-4c0b-bf37-8c9767a55a9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:12.799110 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da1addf-e3bc-4c0b-bf37-8c9767a55a9a admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:12.875613 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 862/1731] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Aug 6 01:33:12 2022] POST /identity/v3/projects => generated 389 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:33:12.880165 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da1addf-e3bc-4c0b-bf37-8c9767a55a9a admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:12.881627 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:33:12.920285 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-31094904-e1a3-42e5-b0d2-c4f95142d608 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28e80afa761e41989ae14fe250b3656c', 'request_id': 'req-31094904-e1a3-42e5-b0d2-c4f95142d608'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:12.930530 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-31094904-e1a3-42e5-b0d2-c4f95142d608 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28e80afa761e41989ae14fe250b3656c', 'request_id': 'req-31094904-e1a3-42e5-b0d2-c4f95142d608'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:12.956291 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-31094904-e1a3-42e5-b0d2-c4f95142d608 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28e80afa761e41989ae14fe250b3656c', 'request_id': 'req-31094904-e1a3-42e5-b0d2-c4f95142d608'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:13.008935 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-31094904-e1a3-42e5-b0d2-c4f95142d608 admin admin] The token cache is being invalidate because project 28e80afa761e41989ae14fe250b3656c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:33:13.008935 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-31094904-e1a3-42e5-b0d2-c4f95142d608 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:13.013291 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-31094904-e1a3-42e5-b0d2-c4f95142d608 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:13.020136 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 870/1732] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Aug 6 01:33:12 2022] DELETE /identity/v3/projects/28e80afa761e41989ae14fe250b3656c => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:33:13.024000 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31094904-e1a3-42e5-b0d2-c4f95142d608 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:13.036830 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-31094904-e1a3-42e5-b0d2-c4f95142d608 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:13.059814 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da1addf-e3bc-4c0b-bf37-8c9767a55a9a admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:13.087859 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:33:13.302054 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31094904-e1a3-42e5-b0d2-c4f95142d608 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:13.316605 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e11958-0ffd-495a-9663-f55a0fb2d564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:13.317712 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e11958-0ffd-495a-9663-f55a0fb2d564 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:13.319247 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e11958-0ffd-495a-9663-f55a0fb2d564 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:13.319689 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e11958-0ffd-495a-9663-f55a0fb2d564 admin admin] PATH_INFO: `/v3/users/d3c183fa7abc4980a56cf07e6d69c459` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:13.321406 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e11958-0ffd-495a-9663-f55a0fb2d564 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3c183fa7abc4980a56cf07e6d69c459)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:13.325550 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e11958-0ffd-495a-9663-f55a0fb2d564 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:13.402642 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f620f0-0bff-44b7-8ef9-cf3b8e67caa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:13.404968 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f620f0-0bff-44b7-8ef9-cf3b8e67caa4 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:13.407002 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f620f0-0bff-44b7-8ef9-cf3b8e67caa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:13.407364 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f620f0-0bff-44b7-8ef9-cf3b8e67caa4 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:13.412313 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f620f0-0bff-44b7-8ef9-cf3b8e67caa4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:13.418891 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f620f0-0bff-44b7-8ef9-cf3b8e67caa4 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:13.446804 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-94e11958-0ffd-495a-9663-f55a0fb2d564 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3c183fa7abc4980a56cf07e6d69c459'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:13.465444 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-94e11958-0ffd-495a-9663-f55a0fb2d564 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3c183fa7abc4980a56cf07e6d69c459'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:13.467238 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-94e11958-0ffd-495a-9663-f55a0fb2d564 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3c183fa7abc4980a56cf07e6d69c459'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:13.470428 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-94e11958-0ffd-495a-9663-f55a0fb2d564 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3c183fa7abc4980a56cf07e6d69c459'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:13.500292 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 863/1733] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:33:12 2022] POST /identity/v3/users => generated 295 bytes in 619 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:33:13.501229 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f620f0-0bff-44b7-8ef9-cf3b8e67caa4 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:13.507435 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-05f620f0-0bff-44b7-8ef9-cf3b8e67caa4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:13.522573 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-94e11958-0ffd-495a-9663-f55a0fb2d564 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3c183fa7abc4980a56cf07e6d69c459'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:13.572540 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 871/1734] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Aug 6 01:33:13 2022] DELETE /identity/v3/users/d3c183fa7abc4980a56cf07e6d69c459 => generated 0 bytes in 546 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:33:13.572540 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e11958-0ffd-495a-9663-f55a0fb2d564 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:13.573965 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-94e11958-0ffd-495a-9663-f55a0fb2d564 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:13.581558 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f620f0-0bff-44b7-8ef9-cf3b8e67caa4 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:13.643995 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:33:13.734134 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e11958-0ffd-495a-9663-f55a0fb2d564 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:13.756299 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e277ae-0830-4003-a523-3abb757dc423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:13.758998 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e277ae-0830-4003-a523-3abb757dc423 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:13.759910 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e277ae-0830-4003-a523-3abb757dc423 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:13.761956 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e277ae-0830-4003-a523-3abb757dc423 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:13.766029 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e277ae-0830-4003-a523-3abb757dc423 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:13.770805 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e277ae-0830-4003-a523-3abb757dc423 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:13.821097 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 872/1735] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:33:13 2022] GET /identity/v3/roles => generated 1420 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:33:13.825563 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e277ae-0830-4003-a523-3abb757dc423 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:13.851648 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409b10d4-baaa-4147-885f-ac93b343ad80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:13.857284 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409b10d4-baaa-4147-885f-ac93b343ad80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:13.858226 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409b10d4-baaa-4147-885f-ac93b343ad80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:13.860594 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409b10d4-baaa-4147-885f-ac93b343ad80 admin admin] PATH_INFO: `/v3/projects/fe39785b9222486eb49e696b67bb2f1f` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:13.865235 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409b10d4-baaa-4147-885f-ac93b343ad80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe39785b9222486eb49e696b67bb2f1f)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:13.870119 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e277ae-0830-4003-a523-3abb757dc423 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:13.874995 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409b10d4-baaa-4147-885f-ac93b343ad80 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:13.888802 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b31899b-301a-4369-bbed-080fbbfda4af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:13.894955 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b31899b-301a-4369-bbed-080fbbfda4af admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:13.895461 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b31899b-301a-4369-bbed-080fbbfda4af admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:13.897374 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b31899b-301a-4369-bbed-080fbbfda4af admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:13.899934 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b31899b-301a-4369-bbed-080fbbfda4af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:13.904595 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b31899b-301a-4369-bbed-080fbbfda4af admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:13.957896 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 873/1736] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:33:13 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:33:13.960826 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b31899b-301a-4369-bbed-080fbbfda4af admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:13.998858 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b31899b-301a-4369-bbed-080fbbfda4af admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:14.012689 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864bc3b3-0d26-4c87-9673-243e021cb517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:14.015349 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864bc3b3-0d26-4c87-9673-243e021cb517 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:14.015894 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864bc3b3-0d26-4c87-9673-243e021cb517 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:14.016345 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864bc3b3-0d26-4c87-9673-243e021cb517 admin admin] PATH_INFO: `/v3/projects/ec1ae00741aa4f8896128bc38ae74d15/users/8bc56f7cbd9546b288735e9ac7e31df8/roles/61b3b93d37a047e4bb8369b46fe20db9` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:14.054589 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864bc3b3-0d26-4c87-9673-243e021cb517 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec1ae00741aa4f8896128bc38ae74d15, user_id=8bc56f7cbd9546b288735e9ac7e31df8, role_id=61b3b93d37a047e4bb8369b46fe20db9)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:14.064551 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864bc3b3-0d26-4c87-9673-243e021cb517 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:14.088768 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 874/1737] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:33:13 2022] PUT /identity/v3/projects/ec1ae00741aa4f8896128bc38ae74d15/users/8bc56f7cbd9546b288735e9ac7e31df8/roles/61b3b93d37a047e4bb8369b46fe20db9 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:33:14.101352 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3282726-1dfd-4aea-be3f-e493cf09f23b None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:14.101754 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3282726-1dfd-4aea-be3f-e493cf09f23b None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:14.102205 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3282726-1dfd-4aea-be3f-e493cf09f23b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:14.140707 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-409b10d4-baaa-4147-885f-ac93b343ad80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe39785b9222486eb49e696b67bb2f1f', 'request_id': 'req-409b10d4-baaa-4147-885f-ac93b343ad80'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:14.188456 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-c3282726-1dfd-4aea-be3f-e493cf09f23b None None] MFA Rules not processed for user `8bc56f7cbd9546b288735e9ac7e31df8`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:33:14.190784 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-c3282726-1dfd-4aea-be3f-e493cf09f23b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:14.192655 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-409b10d4-baaa-4147-885f-ac93b343ad80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe39785b9222486eb49e696b67bb2f1f', 'request_id': 'req-409b10d4-baaa-4147-885f-ac93b343ad80'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:14.240269 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-409b10d4-baaa-4147-885f-ac93b343ad80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe39785b9222486eb49e696b67bb2f1f', 'request_id': 'req-409b10d4-baaa-4147-885f-ac93b343ad80'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:14.304124 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-409b10d4-baaa-4147-885f-ac93b343ad80 admin admin] The token cache is being invalidate because project fe39785b9222486eb49e696b67bb2f1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:33:14.304432 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-409b10d4-baaa-4147-885f-ac93b343ad80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:14.306894 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-409b10d4-baaa-4147-885f-ac93b343ad80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:14.308964 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 864/1738] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Aug 6 01:33:13 2022] DELETE /identity/v3/projects/fe39785b9222486eb49e696b67bb2f1f => generated 0 bytes in 809 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:33:14.317683 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409b10d4-baaa-4147-885f-ac93b343ad80 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:14.333992 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-409b10d4-baaa-4147-885f-ac93b343ad80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:14.487977 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 875/1739] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:33:14 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 387 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:33:14.491201 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3282726-1dfd-4aea-be3f-e493cf09f23b None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:14.498502 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:33:14.501179 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-c3282726-1dfd-4aea-be3f-e493cf09f23b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:14.569284 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409b10d4-baaa-4147-885f-ac93b343ad80 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:14.654149 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e188b2-75f1-49c6-a304-1578b2ec1837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:14.654954 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e188b2-75f1-49c6-a304-1578b2ec1837 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:14.655345 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e188b2-75f1-49c6-a304-1578b2ec1837 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:14.655725 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e188b2-75f1-49c6-a304-1578b2ec1837 admin admin] PATH_INFO: `/v3/users/c25ff9353575471398101a913a863919` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:14.657481 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e188b2-75f1-49c6-a304-1578b2ec1837 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c25ff9353575471398101a913a863919)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:14.660485 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e188b2-75f1-49c6-a304-1578b2ec1837 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:14.782889 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3282726-1dfd-4aea-be3f-e493cf09f23b None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:14.792571 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-04e188b2-75f1-49c6-a304-1578b2ec1837 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c25ff9353575471398101a913a863919'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:14.794487 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-04e188b2-75f1-49c6-a304-1578b2ec1837 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c25ff9353575471398101a913a863919'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:14.796840 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-04e188b2-75f1-49c6-a304-1578b2ec1837 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c25ff9353575471398101a913a863919'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:14.797999 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9077e71b-e621-413f-9714-2db45afab381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:14.808655 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9077e71b-e621-413f-9714-2db45afab381 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:14.809404 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9077e71b-e621-413f-9714-2db45afab381 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:14.812761 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9077e71b-e621-413f-9714-2db45afab381 admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:14.814848 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9077e71b-e621-413f-9714-2db45afab381 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:14.817614 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9077e71b-e621-413f-9714-2db45afab381 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:14.844745 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-04e188b2-75f1-49c6-a304-1578b2ec1837 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c25ff9353575471398101a913a863919'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:14.877059 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-04e188b2-75f1-49c6-a304-1578b2ec1837 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c25ff9353575471398101a913a863919'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:14.923867 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 865/1740] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Aug 6 01:33:14 2022] DELETE /identity/v3/users/c25ff9353575471398101a913a863919 => generated 0 bytes in 604 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:33:14.925494 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 876/1741] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Aug 6 01:33:14 2022] POST /identity/v3/projects => generated 393 bytes in 435 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:33:14.925617 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e188b2-75f1-49c6-a304-1578b2ec1837 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:14.930825 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-04e188b2-75f1-49c6-a304-1578b2ec1837 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:15.162060 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e188b2-75f1-49c6-a304-1578b2ec1837 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:15.200034 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:33:15.347269 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4daada-5c1c-4606-aebf-4b1728a16090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:15.348556 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4daada-5c1c-4606-aebf-4b1728a16090 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:15.349465 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4daada-5c1c-4606-aebf-4b1728a16090 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:15.349961 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4daada-5c1c-4606-aebf-4b1728a16090 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:15.351220 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4daada-5c1c-4606-aebf-4b1728a16090 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:15.356768 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4daada-5c1c-4606-aebf-4b1728a16090 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:15.419374 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 866/1742] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:33:14 2022] POST /identity/v3/users => generated 303 bytes in 495 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:33:15.422514 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9077e71b-e621-413f-9714-2db45afab381 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:15.446153 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9077e71b-e621-413f-9714-2db45afab381 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:15.466721 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e82780a-18c0-47e4-bd6e-5f5fb827932b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:15.467371 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e82780a-18c0-47e4-bd6e-5f5fb827932b admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:15.467655 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e82780a-18c0-47e4-bd6e-5f5fb827932b admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:15.467929 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e82780a-18c0-47e4-bd6e-5f5fb827932b admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:15.468656 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e82780a-18c0-47e4-bd6e-5f5fb827932b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:15.470316 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e82780a-18c0-47e4-bd6e-5f5fb827932b admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:15.506532 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 877/1743] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:33:15 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:33:15.510694 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4daada-5c1c-4606-aebf-4b1728a16090 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:15.527959 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4daada-5c1c-4606-aebf-4b1728a16090 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:15.537298 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9bef0c-ff1d-4a03-a62d-50b5498157c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:15.539106 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9bef0c-ff1d-4a03-a62d-50b5498157c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:15.539106 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9bef0c-ff1d-4a03-a62d-50b5498157c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:15.539424 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9bef0c-ff1d-4a03-a62d-50b5498157c8 admin admin] PATH_INFO: `/v3/projects/fc39a063842a44b490506d5694449996/users/310d15948b1d49f289495877923a92b3/roles/4559b9a0837249f895a13c70096f8d38` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:15.587441 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9bef0c-ff1d-4a03-a62d-50b5498157c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc39a063842a44b490506d5694449996, user_id=310d15948b1d49f289495877923a92b3, role_id=4559b9a0837249f895a13c70096f8d38)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:15.594153 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9bef0c-ff1d-4a03-a62d-50b5498157c8 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:15.631068 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 867/1744] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:33:15 2022] PUT /identity/v3/projects/fc39a063842a44b490506d5694449996/users/310d15948b1d49f289495877923a92b3/roles/4559b9a0837249f895a13c70096f8d38 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:33:15.635127 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e82780a-18c0-47e4-bd6e-5f5fb827932b admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:15.640598 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:33:15.648519 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e82780a-18c0-47e4-bd6e-5f5fb827932b admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:15.777097 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07039da2-c5c2-437c-807f-f2f24fc32708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:15.779307 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07039da2-c5c2-437c-807f-f2f24fc32708 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:15.779862 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07039da2-c5c2-437c-807f-f2f24fc32708 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:15.780152 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07039da2-c5c2-437c-807f-f2f24fc32708 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:15.781435 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07039da2-c5c2-437c-807f-f2f24fc32708 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:15.785647 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07039da2-c5c2-437c-807f-f2f24fc32708 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:15.800333 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 878/1745] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:33:15 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:33:15.804659 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9bef0c-ff1d-4a03-a62d-50b5498157c8 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:15.823374 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9bef0c-ff1d-4a03-a62d-50b5498157c8 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:15.852850 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd25f843-b0a3-4fcf-bb12-466c7ca3f7e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:15.852850 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd25f843-b0a3-4fcf-bb12-466c7ca3f7e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:15.852850 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd25f843-b0a3-4fcf-bb12-466c7ca3f7e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:15.852850 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd25f843-b0a3-4fcf-bb12-466c7ca3f7e6 admin admin] PATH_INFO: `/v3/projects/fc39a063842a44b490506d5694449996/users/310d15948b1d49f289495877923a92b3/roles/4559b9a0837249f895a13c70096f8d38` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:15.854551 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd25f843-b0a3-4fcf-bb12-466c7ca3f7e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc39a063842a44b490506d5694449996, user_id=310d15948b1d49f289495877923a92b3, role_id=4559b9a0837249f895a13c70096f8d38)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:15.861582 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd25f843-b0a3-4fcf-bb12-466c7ca3f7e6 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:15.900040 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 868/1746] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:33:15 2022] PUT /identity/v3/projects/fc39a063842a44b490506d5694449996/users/310d15948b1d49f289495877923a92b3/roles/4559b9a0837249f895a13c70096f8d38 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:33:15.911501 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d01a94-b36a-48c5-b934-eff38a60f85f None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:15.911795 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d01a94-b36a-48c5-b934-eff38a60f85f None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:15.912168 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d01a94-b36a-48c5-b934-eff38a60f85f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:16.014267 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-00d01a94-b36a-48c5-b934-eff38a60f85f None None] MFA Rules not processed for user `310d15948b1d49f289495877923a92b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:33:16.016142 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-00d01a94-b36a-48c5-b934-eff38a60f85f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:16.226862 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd25f843-b0a3-4fcf-bb12-466c7ca3f7e6 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:16.228641 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-dd25f843-b0a3-4fcf-bb12-466c7ca3f7e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:16.236202 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 879/1747] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:33:15 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:33:16.258386 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d01a94-b36a-48c5-b934-eff38a60f85f None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:16.260112 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-00d01a94-b36a-48c5-b934-eff38a60f85f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:16.386719 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd25f843-b0a3-4fcf-bb12-466c7ca3f7e6 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:16.418763 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c052a5fa-25ef-46aa-8ebf-2baaae6160ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:16.419442 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c052a5fa-25ef-46aa-8ebf-2baaae6160ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:16.419714 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c052a5fa-25ef-46aa-8ebf-2baaae6160ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:16.419991 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c052a5fa-25ef-46aa-8ebf-2baaae6160ea admin admin] PATH_INFO: `/v3/projects/02d40069b7174b7d8968b390b2ee3581` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:16.421217 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c052a5fa-25ef-46aa-8ebf-2baaae6160ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=02d40069b7174b7d8968b390b2ee3581)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:16.423485 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c052a5fa-25ef-46aa-8ebf-2baaae6160ea admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:16.450940 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d01a94-b36a-48c5-b934-eff38a60f85f None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:16.464268 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14db29e8-333c-4e54-92cc-b65da676bda5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1be1e6173e374a60b9e1fbf51a1ab571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:16.466697 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14db29e8-333c-4e54-92cc-b65da676bda5 service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:16.466990 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14db29e8-333c-4e54-92cc-b65da676bda5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:16.467273 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14db29e8-333c-4e54-92cc-b65da676bda5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:16.510628 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14db29e8-333c-4e54-92cc-b65da676bda5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:16.512098 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14db29e8-333c-4e54-92cc-b65da676bda5 service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:16.561587 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 880/1748] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:33:16 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:33:16.607829 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-c052a5fa-25ef-46aa-8ebf-2baaae6160ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02d40069b7174b7d8968b390b2ee3581', 'request_id': 'req-c052a5fa-25ef-46aa-8ebf-2baaae6160ea'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:16.624552 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-c052a5fa-25ef-46aa-8ebf-2baaae6160ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02d40069b7174b7d8968b390b2ee3581', 'request_id': 'req-c052a5fa-25ef-46aa-8ebf-2baaae6160ea'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:16.639259 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-c052a5fa-25ef-46aa-8ebf-2baaae6160ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02d40069b7174b7d8968b390b2ee3581', 'request_id': 'req-c052a5fa-25ef-46aa-8ebf-2baaae6160ea'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:16.641147 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=89,keystone:INSERT=5,keystone:DELETE=6 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:33:16.679717 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14db29e8-333c-4e54-92cc-b65da676bda5 service cinder] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:16.685569 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-14db29e8-333c-4e54-92cc-b65da676bda5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:16.686274 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-c052a5fa-25ef-46aa-8ebf-2baaae6160ea admin admin] The token cache is being invalidate because project 02d40069b7174b7d8968b390b2ee3581 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:33:16.686584 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-c052a5fa-25ef-46aa-8ebf-2baaae6160ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:16.689568 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-c052a5fa-25ef-46aa-8ebf-2baaae6160ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:16.696856 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 869/1749] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Aug 6 01:33:16 2022] DELETE /identity/v3/projects/02d40069b7174b7d8968b390b2ee3581 => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:33:16.702598 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09763f62-9d82-434c-be58-bde66bf30ff2 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:16.702895 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09763f62-9d82-434c-be58-bde66bf30ff2 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:16.703176 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09763f62-9d82-434c-be58-bde66bf30ff2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:16.736365 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14db29e8-333c-4e54-92cc-b65da676bda5 service cinder] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:16.754206 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-09763f62-9d82-434c-be58-bde66bf30ff2 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:33:16.755878 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-09763f62-9d82-434c-be58-bde66bf30ff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:16.809199 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:33:16.870284 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2475f7-492c-48ed-8bc3-5983d5af0a9c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1be1e6173e374a60b9e1fbf51a1ab571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:16.871263 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2475f7-492c-48ed-8bc3-5983d5af0a9c service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:16.872201 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2475f7-492c-48ed-8bc3-5983d5af0a9c service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:16.872949 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2475f7-492c-48ed-8bc3-5983d5af0a9c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:16.877280 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-be2475f7-492c-48ed-8bc3-5983d5af0a9c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:16.885411 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 870/1750] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:33:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:33:16.894614 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c743828-db19-484d-aa2e-7c8c4ddbff85 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:16.894897 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c743828-db19-484d-aa2e-7c8c4ddbff85 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:16.895184 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c743828-db19-484d-aa2e-7c8c4ddbff85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:16.926478 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-4c743828-db19-484d-aa2e-7c8c4ddbff85 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:33:16.928867 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-4c743828-db19-484d-aa2e-7c8c4ddbff85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:16.952655 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 871/1751] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:33:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:33:16.960983 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c743828-db19-484d-aa2e-7c8c4ddbff85 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:16.972710 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c743828-db19-484d-aa2e-7c8c4ddbff85 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:16.993397 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9774916f-5ea3-41ff-90a7-603a3980d5f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:16.994042 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9774916f-5ea3-41ff-90a7-603a3980d5f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:16.994323 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9774916f-5ea3-41ff-90a7-603a3980d5f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:16.994601 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9774916f-5ea3-41ff-90a7-603a3980d5f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:16.995319 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9774916f-5ea3-41ff-90a7-603a3980d5f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:33:16.995627 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9774916f-5ea3-41ff-90a7-603a3980d5f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:17.026688 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9774916f-5ea3-41ff-90a7-603a3980d5f1 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:17.069976 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 872/1752] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Aug 6 01:33:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:33:17.075371 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9774916f-5ea3-41ff-90a7-603a3980d5f1 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:17.090201 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2475f7-492c-48ed-8bc3-5983d5af0a9c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:17.094814 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2475f7-492c-48ed-8bc3-5983d5af0a9c service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:17.106289 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9774916f-5ea3-41ff-90a7-603a3980d5f1 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:17.124420 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecf9a23-fe50-4846-916c-d1e23edc4d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:17.125074 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecf9a23-fe50-4846-916c-d1e23edc4d16 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:17.125367 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecf9a23-fe50-4846-916c-d1e23edc4d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:17.125758 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 881/1753] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:33:16 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:33:17.126077 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecf9a23-fe50-4846-916c-d1e23edc4d16 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:17.126971 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecf9a23-fe50-4846-916c-d1e23edc4d16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:17.129685 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecf9a23-fe50-4846-916c-d1e23edc4d16 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:17.172351 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 873/1754] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Aug 6 01:33:17 2022] POST /identity/v3/projects => generated 381 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:33:17.177237 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2475f7-492c-48ed-8bc3-5983d5af0a9c service cinder] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:17.192555 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2475f7-492c-48ed-8bc3-5983d5af0a9c service cinder] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:17.274524 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd429464-3544-4ac3-8921-14d492c93182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:17.276719 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd429464-3544-4ac3-8921-14d492c93182 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:17.278399 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd429464-3544-4ac3-8921-14d492c93182 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:17.280586 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd429464-3544-4ac3-8921-14d492c93182 admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:17.281671 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd429464-3544-4ac3-8921-14d492c93182 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:17.285165 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd429464-3544-4ac3-8921-14d492c93182 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:17.313558 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 882/1755] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:33:17 2022] POST /identity/v3/users => generated 291 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:33:17.318863 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecf9a23-fe50-4846-916c-d1e23edc4d16 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:17.333170 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecf9a23-fe50-4846-916c-d1e23edc4d16 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:17.338831 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c064f9-09e2-421b-bb7c-c1ed64a3a0d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:17.339632 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c064f9-09e2-421b-bb7c-c1ed64a3a0d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:17.339921 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c064f9-09e2-421b-bb7c-c1ed64a3a0d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:17.340927 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c064f9-09e2-421b-bb7c-c1ed64a3a0d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:17.341060 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c064f9-09e2-421b-bb7c-c1ed64a3a0d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:17.342896 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c064f9-09e2-421b-bb7c-c1ed64a3a0d3 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:17.359023 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 874/1756] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:33:17 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:33:17.364622 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd429464-3544-4ac3-8921-14d492c93182 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:17.376752 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd429464-3544-4ac3-8921-14d492c93182 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:17.382249 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef20a00-5385-4f68-bc57-bd8d2349443b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:17.382857 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef20a00-5385-4f68-bc57-bd8d2349443b admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:17.383131 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef20a00-5385-4f68-bc57-bd8d2349443b admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:17.383406 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef20a00-5385-4f68-bc57-bd8d2349443b admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:17.384102 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef20a00-5385-4f68-bc57-bd8d2349443b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:17.387051 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef20a00-5385-4f68-bc57-bd8d2349443b admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:17.402606 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 883/1757] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:33:17 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:33:17.407782 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c064f9-09e2-421b-bb7c-c1ed64a3a0d3 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:17.428252 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c064f9-09e2-421b-bb7c-c1ed64a3a0d3 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:17.435020 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f6d4df-20e3-4552-9c1c-fa33ae4e3409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:17.436082 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f6d4df-20e3-4552-9c1c-fa33ae4e3409 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:17.436418 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f6d4df-20e3-4552-9c1c-fa33ae4e3409 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:17.436703 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f6d4df-20e3-4552-9c1c-fa33ae4e3409 admin admin] PATH_INFO: `/v3/projects/3dd930d9e28248eaa9c7d3eceb4fbc17/users/3f78297230f04491a8e67bc3b649a23c/roles/61b3b93d37a047e4bb8369b46fe20db9` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:17.460534 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f6d4df-20e3-4552-9c1c-fa33ae4e3409 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3dd930d9e28248eaa9c7d3eceb4fbc17, user_id=3f78297230f04491a8e67bc3b649a23c, role_id=61b3b93d37a047e4bb8369b46fe20db9)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:17.464909 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f6d4df-20e3-4552-9c1c-fa33ae4e3409 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:17.493737 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 875/1758] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:33:17 2022] PUT /identity/v3/projects/3dd930d9e28248eaa9c7d3eceb4fbc17/users/3f78297230f04491a8e67bc3b649a23c/roles/61b3b93d37a047e4bb8369b46fe20db9 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:33:17.506118 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d450dc97-78cc-4f9a-8b3b-a51d5c346008 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:17.507059 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d450dc97-78cc-4f9a-8b3b-a51d5c346008 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:17.507549 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d450dc97-78cc-4f9a-8b3b-a51d5c346008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:17.555514 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-d450dc97-78cc-4f9a-8b3b-a51d5c346008 None None] MFA Rules not processed for user `3f78297230f04491a8e67bc3b649a23c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:33:17.556653 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-d450dc97-78cc-4f9a-8b3b-a51d5c346008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:17.581898 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f6d4df-20e3-4552-9c1c-fa33ae4e3409 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:17.583756 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-21f6d4df-20e3-4552-9c1c-fa33ae4e3409 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:17.706118 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:33:17.760020 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 884/1759] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:33:17 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:33:17.765037 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d450dc97-78cc-4f9a-8b3b-a51d5c346008 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:17.833890 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f6d4df-20e3-4552-9c1c-fa33ae4e3409 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:17.845735 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78758dcd-bb5c-4d02-917d-1170588f9120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:17.848594 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78758dcd-bb5c-4d02-917d-1170588f9120 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:17.849070 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78758dcd-bb5c-4d02-917d-1170588f9120 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:17.849518 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78758dcd-bb5c-4d02-917d-1170588f9120 admin admin] PATH_INFO: `/v3/projects/2cbf095db09e439a804134181720a49f` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:17.853746 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78758dcd-bb5c-4d02-917d-1170588f9120 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2cbf095db09e439a804134181720a49f)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:17.858644 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78758dcd-bb5c-4d02-917d-1170588f9120 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:17.907228 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:INSERT=3,keystone:DELETE=1,keystone:SELECT=96 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:33:17.960581 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d450dc97-78cc-4f9a-8b3b-a51d5c346008 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:17.977913 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-78758dcd-bb5c-4d02-917d-1170588f9120 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2cbf095db09e439a804134181720a49f', 'request_id': 'req-78758dcd-bb5c-4d02-917d-1170588f9120'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:17.990333 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-78758dcd-bb5c-4d02-917d-1170588f9120 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2cbf095db09e439a804134181720a49f', 'request_id': 'req-78758dcd-bb5c-4d02-917d-1170588f9120'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:18.004895 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-78758dcd-bb5c-4d02-917d-1170588f9120 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2cbf095db09e439a804134181720a49f', 'request_id': 'req-78758dcd-bb5c-4d02-917d-1170588f9120'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:18.027358 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-78758dcd-bb5c-4d02-917d-1170588f9120 admin admin] The token cache is being invalidate because project 2cbf095db09e439a804134181720a49f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:33:18.028025 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-78758dcd-bb5c-4d02-917d-1170588f9120 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:18.029183 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-78758dcd-bb5c-4d02-917d-1170588f9120 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:18.037729 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 876/1760] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Aug 6 01:33:17 2022] DELETE /identity/v3/projects/2cbf095db09e439a804134181720a49f => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:33:18.075267 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cf581a-9966-441e-8e16-58941834f279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:18.075919 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cf581a-9966-441e-8e16-58941834f279 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:18.076206 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cf581a-9966-441e-8e16-58941834f279 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:18.076483 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cf581a-9966-441e-8e16-58941834f279 admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:18.077191 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cf581a-9966-441e-8e16-58941834f279 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:18.077940 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9156535b-437c-41ef-8e2a-935452833e8d None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:18.079122 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cf581a-9966-441e-8e16-58941834f279 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:18.081119 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9156535b-437c-41ef-8e2a-935452833e8d None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:18.083963 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9156535b-437c-41ef-8e2a-935452833e8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:18.142616 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-9156535b-437c-41ef-8e2a-935452833e8d None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:33:18.144825 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-9156535b-437c-41ef-8e2a-935452833e8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:18.181097 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 885/1761] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Aug 6 01:33:17 2022] POST /identity/v3/projects => generated 379 bytes in 409 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:33:18.181315 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 877/1762] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:33:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:33:18.181403 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cf581a-9966-441e-8e16-58941834f279 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:18.187764 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-a0cf581a-9966-441e-8e16-58941834f279 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:18.188096 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8f045d-ab1b-4db8-9232-109184aedcbd None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:18.188383 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8f045d-ab1b-4db8-9232-109184aedcbd None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:18.188668 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8f045d-ab1b-4db8-9232-109184aedcbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:18.240648 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-bb8f045d-ab1b-4db8-9232-109184aedcbd None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:33:18.241905 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-bb8f045d-ab1b-4db8-9232-109184aedcbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:18.310458 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cf581a-9966-441e-8e16-58941834f279 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:18.327620 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d7d25f-b7ab-4c9e-8609-c221281cb934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:18.328282 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d7d25f-b7ab-4c9e-8609-c221281cb934 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:18.328746 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d7d25f-b7ab-4c9e-8609-c221281cb934 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:18.329032 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d7d25f-b7ab-4c9e-8609-c221281cb934 admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:18.329806 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d7d25f-b7ab-4c9e-8609-c221281cb934 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:18.332654 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d7d25f-b7ab-4c9e-8609-c221281cb934 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:18.396604 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 886/1763] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:33:18 2022] POST /identity/v3/users => generated 296 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:33:18.401298 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d7d25f-b7ab-4c9e-8609-c221281cb934 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:18.442676 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d7d25f-b7ab-4c9e-8609-c221281cb934 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:18.443638 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 878/1764] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:33:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:33:18.447940 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8f045d-ab1b-4db8-9232-109184aedcbd None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:18.456709 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf931ebb-2a6e-441f-b5d6-424185f7e5ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:18.457712 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf931ebb-2a6e-441f-b5d6-424185f7e5ea admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:18.458195 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf931ebb-2a6e-441f-b5d6-424185f7e5ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:18.458764 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf931ebb-2a6e-441f-b5d6-424185f7e5ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:18.459707 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf931ebb-2a6e-441f-b5d6-424185f7e5ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:18.461484 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8f045d-ab1b-4db8-9232-109184aedcbd None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:18.463404 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf931ebb-2a6e-441f-b5d6-424185f7e5ea admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:18.467845 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aab7188-c54f-4ae3-ae69-30836fb65366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:18.468491 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aab7188-c54f-4ae3-ae69-30836fb65366 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:18.468822 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aab7188-c54f-4ae3-ae69-30836fb65366 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:18.469135 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aab7188-c54f-4ae3-ae69-30836fb65366 admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:18.470165 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aab7188-c54f-4ae3-ae69-30836fb65366 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:33:18.470477 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aab7188-c54f-4ae3-ae69-30836fb65366 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:18.472257 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:33:18.472587 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aab7188-c54f-4ae3-ae69-30836fb65366 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:18.486106 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 887/1765] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:33:18 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:33:18.487839 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf931ebb-2a6e-441f-b5d6-424185f7e5ea admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:18.504621 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 879/1766] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Aug 6 01:33:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:33:18.506012 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf931ebb-2a6e-441f-b5d6-424185f7e5ea admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:18.508857 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aab7188-c54f-4ae3-ae69-30836fb65366 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:18.523397 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec98dec-2138-4508-97fe-b888691bf0c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:18.524102 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec98dec-2138-4508-97fe-b888691bf0c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:18.524383 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec98dec-2138-4508-97fe-b888691bf0c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:18.524854 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec98dec-2138-4508-97fe-b888691bf0c7 admin admin] PATH_INFO: `/v3/projects/68aefbc48f0b4b6ab18973acd3cbce30/users/941df4cd428e4d588fe8a5e2ccb58fa7/roles/4559b9a0837249f895a13c70096f8d38` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:18.531364 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aab7188-c54f-4ae3-ae69-30836fb65366 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:18.543086 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1dae55-196c-47e1-bf23-1b8a07555f19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:18.543725 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1dae55-196c-47e1-bf23-1b8a07555f19 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:18.544005 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1dae55-196c-47e1-bf23-1b8a07555f19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:18.544279 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1dae55-196c-47e1-bf23-1b8a07555f19 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:18.544981 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1dae55-196c-47e1-bf23-1b8a07555f19 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:18.546936 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1dae55-196c-47e1-bf23-1b8a07555f19 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:18.550308 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec98dec-2138-4508-97fe-b888691bf0c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68aefbc48f0b4b6ab18973acd3cbce30, user_id=941df4cd428e4d588fe8a5e2ccb58fa7, role_id=4559b9a0837249f895a13c70096f8d38)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:18.555048 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec98dec-2138-4508-97fe-b888691bf0c7 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:18.590247 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 880/1767] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Aug 6 01:33:18 2022] POST /identity/v3/projects => generated 367 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:33:18.595233 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1dae55-196c-47e1-bf23-1b8a07555f19 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:18.601521 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 888/1768] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:33:18 2022] PUT /identity/v3/projects/68aefbc48f0b4b6ab18973acd3cbce30/users/941df4cd428e4d588fe8a5e2ccb58fa7/roles/4559b9a0837249f895a13c70096f8d38 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:33:18.605561 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec98dec-2138-4508-97fe-b888691bf0c7 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:18.691202 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1dae55-196c-47e1-bf23-1b8a07555f19 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:18.710161 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:33:18.721728 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec98dec-2138-4508-97fe-b888691bf0c7 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:18.734882 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59395af-4ff4-446a-a122-9f3d1386bcde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:18.735654 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59395af-4ff4-446a-a122-9f3d1386bcde admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:18.735940 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59395af-4ff4-446a-a122-9f3d1386bcde admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:18.736218 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59395af-4ff4-446a-a122-9f3d1386bcde admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:18.737758 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59395af-4ff4-446a-a122-9f3d1386bcde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:18.741892 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59395af-4ff4-446a-a122-9f3d1386bcde admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:18.754634 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 889/1769] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:33:18 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:33:18.760505 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59395af-4ff4-446a-a122-9f3d1386bcde admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:18.781610 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59395af-4ff4-446a-a122-9f3d1386bcde admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:18.800713 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ead98f-0de3-439c-8588-a89c0cfc92e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:18.801834 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ead98f-0de3-439c-8588-a89c0cfc92e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:18.802187 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ead98f-0de3-439c-8588-a89c0cfc92e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:18.802537 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ead98f-0de3-439c-8588-a89c0cfc92e1 admin admin] PATH_INFO: `/v3/projects/68aefbc48f0b4b6ab18973acd3cbce30/users/941df4cd428e4d588fe8a5e2ccb58fa7/roles/4559b9a0837249f895a13c70096f8d38` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:18.804669 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ead98f-0de3-439c-8588-a89c0cfc92e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68aefbc48f0b4b6ab18973acd3cbce30, user_id=941df4cd428e4d588fe8a5e2ccb58fa7, role_id=4559b9a0837249f895a13c70096f8d38)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:18.807134 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63afce1-22a1-4ad1-9ff0-3ccce06e6503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:18.807445 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ead98f-0de3-439c-8588-a89c0cfc92e1 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:18.807903 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63afce1-22a1-4ad1-9ff0-3ccce06e6503 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:18.808258 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63afce1-22a1-4ad1-9ff0-3ccce06e6503 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:18.808607 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63afce1-22a1-4ad1-9ff0-3ccce06e6503 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:18.810005 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63afce1-22a1-4ad1-9ff0-3ccce06e6503 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:18.812328 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63afce1-22a1-4ad1-9ff0-3ccce06e6503 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:18.822499 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 890/1770] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:33:18 2022] PUT /identity/v3/projects/68aefbc48f0b4b6ab18973acd3cbce30/users/941df4cd428e4d588fe8a5e2ccb58fa7/roles/4559b9a0837249f895a13c70096f8d38 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:33:18.831950 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2c5f28-869f-4c63-b1f4-fa5a2af4d59c None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:18.832241 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2c5f28-869f-4c63-b1f4-fa5a2af4d59c None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:18.832560 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2c5f28-869f-4c63-b1f4-fa5a2af4d59c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:18.852739 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 881/1771] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:33:18 2022] POST /identity/v3/users => generated 284 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:33:18.853748 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63afce1-22a1-4ad1-9ff0-3ccce06e6503 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:18.866057 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63afce1-22a1-4ad1-9ff0-3ccce06e6503 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:18.886813 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-0c2c5f28-869f-4c63-b1f4-fa5a2af4d59c None None] MFA Rules not processed for user `941df4cd428e4d588fe8a5e2ccb58fa7`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:33:18.887817 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-0c2c5f28-869f-4c63-b1f4-fa5a2af4d59c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:18.963632 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3418f0f8-d011-4d6e-9c30-ed94984f1521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:18.964275 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3418f0f8-d011-4d6e-9c30-ed94984f1521 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:18.964576 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3418f0f8-d011-4d6e-9c30-ed94984f1521 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:18.964858 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3418f0f8-d011-4d6e-9c30-ed94984f1521 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:18.965592 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3418f0f8-d011-4d6e-9c30-ed94984f1521 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:18.967199 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3418f0f8-d011-4d6e-9c30-ed94984f1521 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:18.994604 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 882/1772] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:33:18 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:33:18.995537 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3418f0f8-d011-4d6e-9c30-ed94984f1521 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:19.008225 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3418f0f8-d011-4d6e-9c30-ed94984f1521 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:19.009632 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 891/1773] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:33:18 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:33:19.023165 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f0ab90-83ec-43a1-8977-c17d5e93d17d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:19.023800 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f0ab90-83ec-43a1-8977-c17d5e93d17d admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:19.024085 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f0ab90-83ec-43a1-8977-c17d5e93d17d admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:19.024362 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f0ab90-83ec-43a1-8977-c17d5e93d17d admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:19.025081 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f0ab90-83ec-43a1-8977-c17d5e93d17d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:19.026689 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f0ab90-83ec-43a1-8977-c17d5e93d17d admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:19.026878 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2c5f28-869f-4c63-b1f4-fa5a2af4d59c None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:19.028504 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-0c2c5f28-869f-4c63-b1f4-fa5a2af4d59c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:19.041233 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 883/1774] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:33:18 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:33:19.043239 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f0ab90-83ec-43a1-8977-c17d5e93d17d admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:19.055668 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f0ab90-83ec-43a1-8977-c17d5e93d17d admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:19.060754 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443c703b-de35-4aab-adff-6ab472912e7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:19.061482 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443c703b-de35-4aab-adff-6ab472912e7f admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:19.061738 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443c703b-de35-4aab-adff-6ab472912e7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:19.062014 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443c703b-de35-4aab-adff-6ab472912e7f admin admin] PATH_INFO: `/v3/projects/e854c9621e3947a98095554cdcd8c3ba/users/be6ce7c263a14206978a2402d4832a4c/roles/61b3b93d37a047e4bb8369b46fe20db9` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:19.080320 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443c703b-de35-4aab-adff-6ab472912e7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e854c9621e3947a98095554cdcd8c3ba, user_id=be6ce7c263a14206978a2402d4832a4c, role_id=61b3b93d37a047e4bb8369b46fe20db9)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:19.083600 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443c703b-de35-4aab-adff-6ab472912e7f admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:19.094508 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:33:19.105909 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2c5f28-869f-4c63-b1f4-fa5a2af4d59c None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:19.113535 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 884/1775] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:33:19 2022] PUT /identity/v3/projects/e854c9621e3947a98095554cdcd8c3ba/users/be6ce7c263a14206978a2402d4832a4c/roles/61b3b93d37a047e4bb8369b46fe20db9 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:33:19.130966 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbcefda-9a2c-44c3-a367-de23cf3d52bb None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:19.131430 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbcefda-9a2c-44c3-a367-de23cf3d52bb None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:19.131867 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbcefda-9a2c-44c3-a367-de23cf3d52bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:19.166404 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-8bbcefda-9a2c-44c3-a367-de23cf3d52bb None None] MFA Rules not processed for user `be6ce7c263a14206978a2402d4832a4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:33:19.167946 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-8bbcefda-9a2c-44c3-a367-de23cf3d52bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:19.190045 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5999ebd-d382-4712-8e76-f49f359be2c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1be1e6173e374a60b9e1fbf51a1ab571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:19.191266 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5999ebd-d382-4712-8e76-f49f359be2c6 service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:19.191649 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5999ebd-d382-4712-8e76-f49f359be2c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:19.192025 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5999ebd-d382-4712-8e76-f49f359be2c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:19.250865 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:33:19.284544 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 885/1776] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:33:19 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:33:19.285748 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbcefda-9a2c-44c3-a367-de23cf3d52bb None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:19.293334 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5999ebd-d382-4712-8e76-f49f359be2c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:19.295862 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5999ebd-d382-4712-8e76-f49f359be2c6 service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:19.312694 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 892/1777] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:33:19 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:33:19.396709 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbcefda-9a2c-44c3-a367-de23cf3d52bb None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:19.403395 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77556f6b-04c2-4f26-a421-cc5c1ad06b38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:19.404321 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77556f6b-04c2-4f26-a421-cc5c1ad06b38 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:19.404738 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77556f6b-04c2-4f26-a421-cc5c1ad06b38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:19.405183 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77556f6b-04c2-4f26-a421-cc5c1ad06b38 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:19.406070 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77556f6b-04c2-4f26-a421-cc5c1ad06b38 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:19.411375 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77556f6b-04c2-4f26-a421-cc5c1ad06b38 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:19.428834 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5999ebd-d382-4712-8e76-f49f359be2c6 service cinder] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:19.438485 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5999ebd-d382-4712-8e76-f49f359be2c6 service cinder] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:19.452978 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 886/1778] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Aug 6 01:33:19 2022] POST /identity/v3/projects => generated 369 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:33:19.457291 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3a80a5-9db0-401f-88a5-0212e5c200ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1be1e6173e374a60b9e1fbf51a1ab571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:19.457933 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3a80a5-9db0-401f-88a5-0212e5c200ad service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:19.458213 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3a80a5-9db0-401f-88a5-0212e5c200ad service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:19.458491 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3a80a5-9db0-401f-88a5-0212e5c200ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:19.458854 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77556f6b-04c2-4f26-a421-cc5c1ad06b38 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:19.463707 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-7e3a80a5-9db0-401f-88a5-0212e5c200ad service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:19.586166 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3a80a5-9db0-401f-88a5-0212e5c200ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:19.588613 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3a80a5-9db0-401f-88a5-0212e5c200ad service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:19.597897 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77556f6b-04c2-4f26-a421-cc5c1ad06b38 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:19.606696 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61bdb14-8ffe-4407-8f17-f468b2a6f803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:19.607336 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61bdb14-8ffe-4407-8f17-f468b2a6f803 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:19.607623 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61bdb14-8ffe-4407-8f17-f468b2a6f803 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:19.607900 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61bdb14-8ffe-4407-8f17-f468b2a6f803 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:19.608748 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61bdb14-8ffe-4407-8f17-f468b2a6f803 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:19.610538 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61bdb14-8ffe-4407-8f17-f468b2a6f803 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:19.615407 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 893/1779] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:33:19 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 01:33:19.669715 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 887/1780] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:33:19 2022] POST /identity/v3/users => generated 291 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:33:19.672273 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3a80a5-9db0-401f-88a5-0212e5c200ad service cinder] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:19.689267 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3a80a5-9db0-401f-88a5-0212e5c200ad service cinder] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:19.697183 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abc5479-79ac-44e0-84c2-f648a30d1993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:19.698197 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abc5479-79ac-44e0-84c2-f648a30d1993 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:19.698825 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abc5479-79ac-44e0-84c2-f648a30d1993 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:19.699194 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abc5479-79ac-44e0-84c2-f648a30d1993 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:19.700003 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abc5479-79ac-44e0-84c2-f648a30d1993 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:19.702146 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abc5479-79ac-44e0-84c2-f648a30d1993 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:19.716726 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 894/1781] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:33:19 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:33:19.722548 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61bdb14-8ffe-4407-8f17-f468b2a6f803 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:19.742803 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61bdb14-8ffe-4407-8f17-f468b2a6f803 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:19.753676 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ffcace-99f3-4690-8d5b-ea144bfb5e22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:19.756460 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ffcace-99f3-4690-8d5b-ea144bfb5e22 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:19.756755 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ffcace-99f3-4690-8d5b-ea144bfb5e22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:19.757252 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ffcace-99f3-4690-8d5b-ea144bfb5e22 admin admin] PATH_INFO: `/v3/projects/1ead2b3e0775428eadabddb4256c5b3d/users/ad7d276321c54bf993f6d7c4ff3a6ef5/roles/4559b9a0837249f895a13c70096f8d38` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:19.782839 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ffcace-99f3-4690-8d5b-ea144bfb5e22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ead2b3e0775428eadabddb4256c5b3d, user_id=ad7d276321c54bf993f6d7c4ff3a6ef5, role_id=4559b9a0837249f895a13c70096f8d38)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:19.785656 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ffcace-99f3-4690-8d5b-ea144bfb5e22 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:19.816168 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abc5479-79ac-44e0-84c2-f648a30d1993 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:19.817087 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 888/1782] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:33:19 2022] PUT /identity/v3/projects/1ead2b3e0775428eadabddb4256c5b3d/users/ad7d276321c54bf993f6d7c4ff3a6ef5/roles/4559b9a0837249f895a13c70096f8d38 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:33:19.840669 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abc5479-79ac-44e0-84c2-f648a30d1993 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:19.917705 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:33:19.973210 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b190f9-4837-4f5d-949f-04eafb401f1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:19.974523 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b190f9-4837-4f5d-949f-04eafb401f1a admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:19.974942 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b190f9-4837-4f5d-949f-04eafb401f1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:19.975371 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b190f9-4837-4f5d-949f-04eafb401f1a admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:19.980968 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b190f9-4837-4f5d-949f-04eafb401f1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:19.991496 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b190f9-4837-4f5d-949f-04eafb401f1a admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:20.029965 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 895/1783] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:33:19 2022] GET /identity/v3/roles => generated 1420 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:33:20.035216 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ffcace-99f3-4690-8d5b-ea144bfb5e22 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:20.048177 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ffcace-99f3-4690-8d5b-ea144bfb5e22 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:20.058351 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3c7285-68bf-48d0-b408-8c7ed737eb05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:20.059491 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3c7285-68bf-48d0-b408-8c7ed737eb05 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:20.059491 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3c7285-68bf-48d0-b408-8c7ed737eb05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:20.059680 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3c7285-68bf-48d0-b408-8c7ed737eb05 admin admin] PATH_INFO: `/v3/projects/1ead2b3e0775428eadabddb4256c5b3d/users/ad7d276321c54bf993f6d7c4ff3a6ef5/roles/4559b9a0837249f895a13c70096f8d38` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:20.065574 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3c7285-68bf-48d0-b408-8c7ed737eb05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ead2b3e0775428eadabddb4256c5b3d, user_id=ad7d276321c54bf993f6d7c4ff3a6ef5, role_id=4559b9a0837249f895a13c70096f8d38)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:20.072353 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3c7285-68bf-48d0-b408-8c7ed737eb05 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:20.113198 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 889/1784] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:33:20 2022] PUT /identity/v3/projects/1ead2b3e0775428eadabddb4256c5b3d/users/ad7d276321c54bf993f6d7c4ff3a6ef5/roles/4559b9a0837249f895a13c70096f8d38 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:33:20.123878 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903b3f35-5652-4a99-a5d7-e586c6943ddb None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:20.124430 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903b3f35-5652-4a99-a5d7-e586c6943ddb None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:20.124725 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903b3f35-5652-4a99-a5d7-e586c6943ddb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:20.182113 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-903b3f35-5652-4a99-a5d7-e586c6943ddb None None] MFA Rules not processed for user `ad7d276321c54bf993f6d7c4ff3a6ef5`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:33:20.185203 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-903b3f35-5652-4a99-a5d7-e586c6943ddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:20.369689 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 896/1785] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:33:20 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:33:20.442031 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3c7285-68bf-48d0-b408-8c7ed737eb05 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:20.444923 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-8a3c7285-68bf-48d0-b408-8c7ed737eb05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:20.485201 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:33:20.522559 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903b3f35-5652-4a99-a5d7-e586c6943ddb None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:20.542080 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3c7285-68bf-48d0-b408-8c7ed737eb05 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:20.547718 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc46220a-d077-4e69-9176-7f19e920321a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1be1e6173e374a60b9e1fbf51a1ab571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:20.548580 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc46220a-d077-4e69-9176-7f19e920321a service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:20.548874 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc46220a-d077-4e69-9176-7f19e920321a service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:20.549182 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc46220a-d077-4e69-9176-7f19e920321a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:20.563109 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc46220a-d077-4e69-9176-7f19e920321a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:20.565409 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc46220a-d077-4e69-9176-7f19e920321a service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:20.580274 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 890/1786] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:33:20 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:33:20.635627 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903b3f35-5652-4a99-a5d7-e586c6943ddb None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:20.641930 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b181010-6258-4425-8fc3-6faeed43320e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:20.642649 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b181010-6258-4425-8fc3-6faeed43320e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:20.642934 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b181010-6258-4425-8fc3-6faeed43320e admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:20.643269 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b181010-6258-4425-8fc3-6faeed43320e admin admin] PATH_INFO: `/v3/users/3f78297230f04491a8e67bc3b649a23c` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:20.645153 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b181010-6258-4425-8fc3-6faeed43320e admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f78297230f04491a8e67bc3b649a23c)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:20.647089 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b181010-6258-4425-8fc3-6faeed43320e admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:20.670719 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc46220a-d077-4e69-9176-7f19e920321a service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:20.682547 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc46220a-d077-4e69-9176-7f19e920321a service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:20.687836 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47496415-800c-4ac9-a94b-cb9cf27d79b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1be1e6173e374a60b9e1fbf51a1ab571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:20.689177 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47496415-800c-4ac9-a94b-cb9cf27d79b1 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:20.689566 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47496415-800c-4ac9-a94b-cb9cf27d79b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:20.690339 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47496415-800c-4ac9-a94b-cb9cf27d79b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:20.722690 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-8b181010-6258-4425-8fc3-6faeed43320e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f78297230f04491a8e67bc3b649a23c'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:20.731776 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-8b181010-6258-4425-8fc3-6faeed43320e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f78297230f04491a8e67bc3b649a23c'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:20.732728 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-8b181010-6258-4425-8fc3-6faeed43320e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f78297230f04491a8e67bc3b649a23c'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:20.733531 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-8b181010-6258-4425-8fc3-6faeed43320e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f78297230f04491a8e67bc3b649a23c'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:20.740571 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:33:20.755802 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-8b181010-6258-4425-8fc3-6faeed43320e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f78297230f04491a8e67bc3b649a23c'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:20.787587 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 897/1787] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Aug 6 01:33:20 2022] DELETE /identity/v3/users/3f78297230f04491a8e67bc3b649a23c => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:33:20.792329 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb301fe-8bdf-4ff5-ab32-fd73e75622e5 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:20.793607 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47496415-800c-4ac9-a94b-cb9cf27d79b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:20.794159 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb301fe-8bdf-4ff5-ab32-fd73e75622e5 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:20.794835 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb301fe-8bdf-4ff5-ab32-fd73e75622e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:20.795521 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47496415-800c-4ac9-a94b-cb9cf27d79b1 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:20.821857 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 891/1788] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:33:20 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 01:33:20.852280 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-dcb301fe-8bdf-4ff5-ab32-fd73e75622e5 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:33:20.858416 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-dcb301fe-8bdf-4ff5-ab32-fd73e75622e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:21.030893 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 898/1789] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:33:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:33:21.086528 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47496415-800c-4ac9-a94b-cb9cf27d79b1 service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:21.088203 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-47496415-800c-4ac9-a94b-cb9cf27d79b1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:21.185785 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47496415-800c-4ac9-a94b-cb9cf27d79b1 service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:21.326311 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9062a68a-b279-4f62-a73d-52cccf348e8c None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:21.326613 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9062a68a-b279-4f62-a73d-52cccf348e8c None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:21.327959 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9062a68a-b279-4f62-a73d-52cccf348e8c None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:21.329252 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 899/1790] 10.222.0.26 () {54 vars in 921 bytes} [Sat Aug 6 01:33:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 01:33:21.331519 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed1249d-3abc-49bc-8f02-99aaceee5da6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:21.333085 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed1249d-3abc-49bc-8f02-99aaceee5da6 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:21.333608 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed1249d-3abc-49bc-8f02-99aaceee5da6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:21.334097 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed1249d-3abc-49bc-8f02-99aaceee5da6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:21.347942 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed1249d-3abc-49bc-8f02-99aaceee5da6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:21.350229 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed1249d-3abc-49bc-8f02-99aaceee5da6 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:21.369388 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6bee06-1ddd-47b0-8366-9b99cc0a717e None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:21.369679 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6bee06-1ddd-47b0-8366-9b99cc0a717e None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:21.371198 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6bee06-1ddd-47b0-8366-9b99cc0a717e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:21.380753 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 892/1791] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:33:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:33:21.402716 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-5e6bee06-1ddd-47b0-8366-9b99cc0a717e None None] MFA Rules not processed for user `b87adc1d5bc240e99f95f6595b350be4`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:33:21.403861 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-5e6bee06-1ddd-47b0-8366-9b99cc0a717e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:21.500177 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 900/1792] 10.222.0.26 () {58 vars in 1018 bytes} [Sat Aug 6 01:33:21 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:33:21.634707 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed1249d-3abc-49bc-8f02-99aaceee5da6 service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:21.636199 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-2ed1249d-3abc-49bc-8f02-99aaceee5da6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:21.661108 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed1249d-3abc-49bc-8f02-99aaceee5da6 service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:21.667205 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962fb760-06d2-45ed-a1f9-ada4f279dbf6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87adc1d5bc240e99f95f6595b350be4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:21.667927 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962fb760-06d2-45ed-a1f9-ada4f279dbf6 service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:21.668289 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962fb760-06d2-45ed-a1f9-ada4f279dbf6 service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:21.668758 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962fb760-06d2-45ed-a1f9-ada4f279dbf6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:21.678922 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962fb760-06d2-45ed-a1f9-ada4f279dbf6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:21.680610 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962fb760-06d2-45ed-a1f9-ada4f279dbf6 service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:21.692418 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 893/1793] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:33:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:33:21.927660 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6bee06-1ddd-47b0-8366-9b99cc0a717e None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:21.929174 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-5e6bee06-1ddd-47b0-8366-9b99cc0a717e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:21.944272 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6bee06-1ddd-47b0-8366-9b99cc0a717e None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:21.948893 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b770f86-f935-4760-95eb-a9db7ff113f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:21.949577 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b770f86-f935-4760-95eb-a9db7ff113f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:21.949858 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b770f86-f935-4760-95eb-a9db7ff113f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:21.950270 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b770f86-f935-4760-95eb-a9db7ff113f6 admin admin] PATH_INFO: `/v3/projects/3dd930d9e28248eaa9c7d3eceb4fbc17` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:21.951473 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b770f86-f935-4760-95eb-a9db7ff113f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3dd930d9e28248eaa9c7d3eceb4fbc17)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:21.953535 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b770f86-f935-4760-95eb-a9db7ff113f6 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:22.007628 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-9b770f86-f935-4760-95eb-a9db7ff113f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3dd930d9e28248eaa9c7d3eceb4fbc17', 'request_id': 'req-9b770f86-f935-4760-95eb-a9db7ff113f6'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:22.014025 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-9b770f86-f935-4760-95eb-a9db7ff113f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3dd930d9e28248eaa9c7d3eceb4fbc17', 'request_id': 'req-9b770f86-f935-4760-95eb-a9db7ff113f6'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:22.019887 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-9b770f86-f935-4760-95eb-a9db7ff113f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3dd930d9e28248eaa9c7d3eceb4fbc17', 'request_id': 'req-9b770f86-f935-4760-95eb-a9db7ff113f6'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:22.022493 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:33:22.036487 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-9b770f86-f935-4760-95eb-a9db7ff113f6 admin admin] The token cache is being invalidate because project 3dd930d9e28248eaa9c7d3eceb4fbc17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:33:22.036784 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-9b770f86-f935-4760-95eb-a9db7ff113f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:22.037442 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-9b770f86-f935-4760-95eb-a9db7ff113f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:22.039238 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 901/1794] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Aug 6 01:33:21 2022] DELETE /identity/v3/projects/3dd930d9e28248eaa9c7d3eceb4fbc17 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:33:22.043208 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962fb760-06d2-45ed-a1f9-ada4f279dbf6 service nova] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:22.060887 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962fb760-06d2-45ed-a1f9-ada4f279dbf6 service nova] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:22.078502 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:33:22.152258 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aacfe40-6fdc-4396-9793-f9bb32501899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:22.152937 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aacfe40-6fdc-4396-9793-f9bb32501899 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:22.153300 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aacfe40-6fdc-4396-9793-f9bb32501899 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:22.153593 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aacfe40-6fdc-4396-9793-f9bb32501899 admin admin] PATH_INFO: `/v3/users/941df4cd428e4d588fe8a5e2ccb58fa7` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:22.154838 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aacfe40-6fdc-4396-9793-f9bb32501899 admin admin] RBAC: Authorizing `identity:delete_user(user_id=941df4cd428e4d588fe8a5e2ccb58fa7)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:22.156982 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aacfe40-6fdc-4396-9793-f9bb32501899 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:22.232108 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-0aacfe40-6fdc-4396-9793-f9bb32501899 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '941df4cd428e4d588fe8a5e2ccb58fa7'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:22.232774 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-0aacfe40-6fdc-4396-9793-f9bb32501899 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '941df4cd428e4d588fe8a5e2ccb58fa7'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:22.233401 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-0aacfe40-6fdc-4396-9793-f9bb32501899 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '941df4cd428e4d588fe8a5e2ccb58fa7'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:22.247282 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-0aacfe40-6fdc-4396-9793-f9bb32501899 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '941df4cd428e4d588fe8a5e2ccb58fa7'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:22.255871 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-0aacfe40-6fdc-4396-9793-f9bb32501899 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '941df4cd428e4d588fe8a5e2ccb58fa7'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:22.279343 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 894/1795] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Aug 6 01:33:22 2022] DELETE /identity/v3/users/941df4cd428e4d588fe8a5e2ccb58fa7 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:33:22.767248 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b770f86-f935-4760-95eb-a9db7ff113f6 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:22.773900 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-9b770f86-f935-4760-95eb-a9db7ff113f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:22.806449 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b770f86-f935-4760-95eb-a9db7ff113f6 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:22.945729 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ab2829-dfbe-4606-a7df-978f376f4a4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:22.947165 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ab2829-dfbe-4606-a7df-978f376f4a4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:22.952645 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ab2829-dfbe-4606-a7df-978f376f4a4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:22.953021 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ab2829-dfbe-4606-a7df-978f376f4a4a admin admin] PATH_INFO: `/v3/users/8bc56f7cbd9546b288735e9ac7e31df8` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:22.956052 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ab2829-dfbe-4606-a7df-978f376f4a4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8bc56f7cbd9546b288735e9ac7e31df8)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:22.958061 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ab2829-dfbe-4606-a7df-978f376f4a4a admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:23.073814 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-19ab2829-dfbe-4606-a7df-978f376f4a4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8bc56f7cbd9546b288735e9ac7e31df8'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:23.074868 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-19ab2829-dfbe-4606-a7df-978f376f4a4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8bc56f7cbd9546b288735e9ac7e31df8'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:23.075849 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-19ab2829-dfbe-4606-a7df-978f376f4a4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8bc56f7cbd9546b288735e9ac7e31df8'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:23.076617 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-19ab2829-dfbe-4606-a7df-978f376f4a4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8bc56f7cbd9546b288735e9ac7e31df8'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:23.106197 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-19ab2829-dfbe-4606-a7df-978f376f4a4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8bc56f7cbd9546b288735e9ac7e31df8'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:23.131305 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 902/1796] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Aug 6 01:33:22 2022] DELETE /identity/v3/users/8bc56f7cbd9546b288735e9ac7e31df8 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:33:23.132869 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba69867-70a6-4d98-9d5c-89af8aec065c None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:23.133270 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba69867-70a6-4d98-9d5c-89af8aec065c None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:23.133455 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba69867-70a6-4d98-9d5c-89af8aec065c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:23.157721 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-3ba69867-70a6-4d98-9d5c-89af8aec065c None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:33:23.159933 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-3ba69867-70a6-4d98-9d5c-89af8aec065c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:23.245997 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 895/1797] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:33:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:33:23.307038 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ab2829-dfbe-4606-a7df-978f376f4a4a admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:23.309294 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-19ab2829-dfbe-4606-a7df-978f376f4a4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:23.400675 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ab2829-dfbe-4606-a7df-978f376f4a4a admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:23.404633 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8486e681-423c-410a-b605-7ec42db4a706 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:23.405306 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8486e681-423c-410a-b605-7ec42db4a706 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:23.405583 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8486e681-423c-410a-b605-7ec42db4a706 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:23.405882 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8486e681-423c-410a-b605-7ec42db4a706 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:23.428565 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8486e681-423c-410a-b605-7ec42db4a706 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:23.430095 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8486e681-423c-410a-b605-7ec42db4a706 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:23.440856 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 903/1798] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:33:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:33:23.827667 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba69867-70a6-4d98-9d5c-89af8aec065c None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:23.829149 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-3ba69867-70a6-4d98-9d5c-89af8aec065c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:23.838655 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8486e681-423c-410a-b605-7ec42db4a706 service neutron] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:23.839995 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-8486e681-423c-410a-b605-7ec42db4a706 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:23.868839 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba69867-70a6-4d98-9d5c-89af8aec065c None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:23.873765 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8486e681-423c-410a-b605-7ec42db4a706 service neutron] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:23.880804 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d481053-e3f7-4a07-96cc-c4a36b9a66f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:23.881505 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d481053-e3f7-4a07-96cc-c4a36b9a66f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:23.881787 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d481053-e3f7-4a07-96cc-c4a36b9a66f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:23.882108 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d481053-e3f7-4a07-96cc-c4a36b9a66f4 admin admin] PATH_INFO: `/v3/projects/68aefbc48f0b4b6ab18973acd3cbce30` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:23.883428 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d481053-e3f7-4a07-96cc-c4a36b9a66f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=68aefbc48f0b4b6ab18973acd3cbce30)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:23.885500 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d481053-e3f7-4a07-96cc-c4a36b9a66f4 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:23.887319 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ece0fd-4ea1-41a1-8c3e-df710ceb8db3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:23.888486 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ece0fd-4ea1-41a1-8c3e-df710ceb8db3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:23.890552 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ece0fd-4ea1-41a1-8c3e-df710ceb8db3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:23.891050 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ece0fd-4ea1-41a1-8c3e-df710ceb8db3 admin admin] PATH_INFO: `/v3/projects/ec1ae00741aa4f8896128bc38ae74d15` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:23.893930 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ece0fd-4ea1-41a1-8c3e-df710ceb8db3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec1ae00741aa4f8896128bc38ae74d15)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:23.898754 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:33:23.901245 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ece0fd-4ea1-41a1-8c3e-df710ceb8db3 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:23.963546 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:33:23.969064 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-0d481053-e3f7-4a07-96cc-c4a36b9a66f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68aefbc48f0b4b6ab18973acd3cbce30', 'request_id': 'req-0d481053-e3f7-4a07-96cc-c4a36b9a66f4'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:23.980019 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-0d481053-e3f7-4a07-96cc-c4a36b9a66f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68aefbc48f0b4b6ab18973acd3cbce30', 'request_id': 'req-0d481053-e3f7-4a07-96cc-c4a36b9a66f4'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:23.987538 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-20ece0fd-4ea1-41a1-8c3e-df710ceb8db3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec1ae00741aa4f8896128bc38ae74d15', 'request_id': 'req-20ece0fd-4ea1-41a1-8c3e-df710ceb8db3'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:23.989299 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-0d481053-e3f7-4a07-96cc-c4a36b9a66f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68aefbc48f0b4b6ab18973acd3cbce30', 'request_id': 'req-0d481053-e3f7-4a07-96cc-c4a36b9a66f4'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:23.998589 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-20ece0fd-4ea1-41a1-8c3e-df710ceb8db3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec1ae00741aa4f8896128bc38ae74d15', 'request_id': 'req-20ece0fd-4ea1-41a1-8c3e-df710ceb8db3'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:24.007051 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-20ece0fd-4ea1-41a1-8c3e-df710ceb8db3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec1ae00741aa4f8896128bc38ae74d15', 'request_id': 'req-20ece0fd-4ea1-41a1-8c3e-df710ceb8db3'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:24.012003 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-0d481053-e3f7-4a07-96cc-c4a36b9a66f4 admin admin] The token cache is being invalidate because project 68aefbc48f0b4b6ab18973acd3cbce30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:33:24.012329 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-0d481053-e3f7-4a07-96cc-c4a36b9a66f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:24.013444 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-0d481053-e3f7-4a07-96cc-c4a36b9a66f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:24.016981 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 904/1799] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Aug 6 01:33:23 2022] DELETE /identity/v3/projects/68aefbc48f0b4b6ab18973acd3cbce30 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:33:24.023282 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c5bb48-ab78-4a18-92d9-b69a31d07bbc None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:24.023723 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c5bb48-ab78-4a18-92d9-b69a31d07bbc None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:24.024022 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c5bb48-ab78-4a18-92d9-b69a31d07bbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:24.035330 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-20ece0fd-4ea1-41a1-8c3e-df710ceb8db3 admin admin] The token cache is being invalidate because project ec1ae00741aa4f8896128bc38ae74d15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:33:24.036680 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-20ece0fd-4ea1-41a1-8c3e-df710ceb8db3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:24.038185 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-20ece0fd-4ea1-41a1-8c3e-df710ceb8db3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:24.040624 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 896/1800] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Aug 6 01:33:23 2022] DELETE /identity/v3/projects/ec1ae00741aa4f8896128bc38ae74d15 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:33:24.044455 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ece0fd-4ea1-41a1-8c3e-df710ceb8db3 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:24.048386 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-20ece0fd-4ea1-41a1-8c3e-df710ceb8db3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:24.062768 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-10c5bb48-ab78-4a18-92d9-b69a31d07bbc None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:33:24.063965 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-10c5bb48-ab78-4a18-92d9-b69a31d07bbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:24.143406 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ece0fd-4ea1-41a1-8c3e-df710ceb8db3 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:24.153383 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d728e25-3b7f-4411-96e7-0e9cac6b0efc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:24.154492 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d728e25-3b7f-4411-96e7-0e9cac6b0efc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:24.155094 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d728e25-3b7f-4411-96e7-0e9cac6b0efc admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:24.155590 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d728e25-3b7f-4411-96e7-0e9cac6b0efc admin admin] PATH_INFO: `/v3/users/310d15948b1d49f289495877923a92b3` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:24.157974 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d728e25-3b7f-4411-96e7-0e9cac6b0efc admin admin] RBAC: Authorizing `identity:delete_user(user_id=310d15948b1d49f289495877923a92b3)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:24.162133 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d728e25-3b7f-4411-96e7-0e9cac6b0efc admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:24.221570 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 905/1801] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:33:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:33:24.227467 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379d0cc8-0dc6-484b-b524-c04bf941f322 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:24.228267 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379d0cc8-0dc6-484b-b524-c04bf941f322 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:24.228551 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379d0cc8-0dc6-484b-b524-c04bf941f322 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:24.249900 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-4d728e25-3b7f-4411-96e7-0e9cac6b0efc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '310d15948b1d49f289495877923a92b3'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:24.250763 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-4d728e25-3b7f-4411-96e7-0e9cac6b0efc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '310d15948b1d49f289495877923a92b3'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:24.254207 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-4d728e25-3b7f-4411-96e7-0e9cac6b0efc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '310d15948b1d49f289495877923a92b3'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:24.266146 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-379d0cc8-0dc6-484b-b524-c04bf941f322 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:33:24.267308 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-379d0cc8-0dc6-484b-b524-c04bf941f322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:24.281459 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-4d728e25-3b7f-4411-96e7-0e9cac6b0efc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '310d15948b1d49f289495877923a92b3'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:24.286664 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 906/1802] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:33:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:33:24.291477 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379d0cc8-0dc6-484b-b524-c04bf941f322 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:24.301261 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-4d728e25-3b7f-4411-96e7-0e9cac6b0efc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '310d15948b1d49f289495877923a92b3'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:24.305230 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379d0cc8-0dc6-484b-b524-c04bf941f322 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:24.312601 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab7d31a-794d-4d79-bb8e-5f609fae84ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:24.313629 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab7d31a-794d-4d79-bb8e-5f609fae84ff admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:24.314123 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab7d31a-794d-4d79-bb8e-5f609fae84ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:24.314586 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab7d31a-794d-4d79-bb8e-5f609fae84ff admin admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:24.315861 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab7d31a-794d-4d79-bb8e-5f609fae84ff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:33:24.316883 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab7d31a-794d-4d79-bb8e-5f609fae84ff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:24.321509 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 897/1803] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Aug 6 01:33:24 2022] DELETE /identity/v3/users/310d15948b1d49f289495877923a92b3 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:33:24.322371 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab7d31a-794d-4d79-bb8e-5f609fae84ff admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:24.337040 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 907/1804] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Aug 6 01:33:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:33:24.342104 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d728e25-3b7f-4411-96e7-0e9cac6b0efc admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:24.393018 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:33:24.429502 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d728e25-3b7f-4411-96e7-0e9cac6b0efc admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:24.435071 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f6b458-f9b2-41ea-9b77-d1dc6834e1a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:24.435886 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f6b458-f9b2-41ea-9b77-d1dc6834e1a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:24.436741 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f6b458-f9b2-41ea-9b77-d1dc6834e1a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:24.437028 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f6b458-f9b2-41ea-9b77-d1dc6834e1a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:24.437875 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f6b458-f9b2-41ea-9b77-d1dc6834e1a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:24.439929 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f6b458-f9b2-41ea-9b77-d1dc6834e1a8 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:24.479474 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 898/1805] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Aug 6 01:33:24 2022] POST /identity/v3/projects => generated 373 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:33:24.480584 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab7d31a-794d-4d79-bb8e-5f609fae84ff admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:24.495180 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab7d31a-794d-4d79-bb8e-5f609fae84ff admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:24.573550 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799a8dd2-8041-4b7d-964f-9b6e6fe24577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:24.574950 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799a8dd2-8041-4b7d-964f-9b6e6fe24577 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:24.575385 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799a8dd2-8041-4b7d-964f-9b6e6fe24577 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:24.575807 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799a8dd2-8041-4b7d-964f-9b6e6fe24577 admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:24.576609 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799a8dd2-8041-4b7d-964f-9b6e6fe24577 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:24.578736 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799a8dd2-8041-4b7d-964f-9b6e6fe24577 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:24.607959 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 908/1806] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:33:24 2022] POST /identity/v3/users => generated 287 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:33:24.612524 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f6b458-f9b2-41ea-9b77-d1dc6834e1a8 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:24.625603 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f6b458-f9b2-41ea-9b77-d1dc6834e1a8 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:24.632717 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19e546b-72fb-4875-a86a-23ee2b3ecb7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:24.633504 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19e546b-72fb-4875-a86a-23ee2b3ecb7e admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:24.633788 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19e546b-72fb-4875-a86a-23ee2b3ecb7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:24.634067 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19e546b-72fb-4875-a86a-23ee2b3ecb7e admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:24.634819 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19e546b-72fb-4875-a86a-23ee2b3ecb7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:24.636550 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19e546b-72fb-4875-a86a-23ee2b3ecb7e admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:24.649840 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 899/1807] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:33:24 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:33:24.657252 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799a8dd2-8041-4b7d-964f-9b6e6fe24577 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:24.662579 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19e546b-72fb-4875-a86a-23ee2b3ecb7e admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:24.665381 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-a19e546b-72fb-4875-a86a-23ee2b3ecb7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:24.670657 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799a8dd2-8041-4b7d-964f-9b6e6fe24577 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:24.676699 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d696fc3-4a20-4d63-bd13-b636fce758fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:24.677424 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d696fc3-4a20-4d63-bd13-b636fce758fc admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:24.686788 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19e546b-72fb-4875-a86a-23ee2b3ecb7e admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:24.687706 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d696fc3-4a20-4d63-bd13-b636fce758fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:24.688136 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d696fc3-4a20-4d63-bd13-b636fce758fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:24.689391 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d696fc3-4a20-4d63-bd13-b636fce758fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:24.691075 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d696fc3-4a20-4d63-bd13-b636fce758fc admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:24.694461 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:33:24.709022 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 909/1808] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:33:24 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 01:33:24.712305 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dd34c4-50c9-4b9f-9663-8f83a7259587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:24.712894 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d696fc3-4a20-4d63-bd13-b636fce758fc admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:24.713110 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dd34c4-50c9-4b9f-9663-8f83a7259587 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:24.713376 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dd34c4-50c9-4b9f-9663-8f83a7259587 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:24.713848 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dd34c4-50c9-4b9f-9663-8f83a7259587 admin admin] PATH_INFO: `/v3/projects/fc39a063842a44b490506d5694449996` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:24.715170 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dd34c4-50c9-4b9f-9663-8f83a7259587 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc39a063842a44b490506d5694449996)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:24.720380 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dd34c4-50c9-4b9f-9663-8f83a7259587 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:24.731345 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d696fc3-4a20-4d63-bd13-b636fce758fc admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:24.736230 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109104ad-ad74-4d87-a112-476e3a5135e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:24.738052 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109104ad-ad74-4d87-a112-476e3a5135e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:24.738345 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109104ad-ad74-4d87-a112-476e3a5135e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:24.738976 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109104ad-ad74-4d87-a112-476e3a5135e8 admin admin] PATH_INFO: `/v3/projects/d37acbae2bf64dac8eec9721189268d0/users/3f081e5336f74e21903f11df0b9d95b1/roles/61b3b93d37a047e4bb8369b46fe20db9` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:24.760630 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109104ad-ad74-4d87-a112-476e3a5135e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d37acbae2bf64dac8eec9721189268d0, user_id=3f081e5336f74e21903f11df0b9d95b1, role_id=61b3b93d37a047e4bb8369b46fe20db9)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:24.777415 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109104ad-ad74-4d87-a112-476e3a5135e8 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:24.797535 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 910/1809] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:33:24 2022] PUT /identity/v3/projects/d37acbae2bf64dac8eec9721189268d0/users/3f081e5336f74e21903f11df0b9d95b1/roles/61b3b93d37a047e4bb8369b46fe20db9 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:33:24.803688 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1840ee63-cf91-4efc-83d4-7620b3c63e6a None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:24.803978 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1840ee63-cf91-4efc-83d4-7620b3c63e6a None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:24.804256 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1840ee63-cf91-4efc-83d4-7620b3c63e6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:24.811011 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-d4dd34c4-50c9-4b9f-9663-8f83a7259587 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc39a063842a44b490506d5694449996', 'request_id': 'req-d4dd34c4-50c9-4b9f-9663-8f83a7259587'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:24.818828 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-d4dd34c4-50c9-4b9f-9663-8f83a7259587 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc39a063842a44b490506d5694449996', 'request_id': 'req-d4dd34c4-50c9-4b9f-9663-8f83a7259587'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:24.832396 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-d4dd34c4-50c9-4b9f-9663-8f83a7259587 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc39a063842a44b490506d5694449996', 'request_id': 'req-d4dd34c4-50c9-4b9f-9663-8f83a7259587'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:24.841215 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-1840ee63-cf91-4efc-83d4-7620b3c63e6a None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:33:24.843002 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-1840ee63-cf91-4efc-83d4-7620b3c63e6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:24.855179 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-d4dd34c4-50c9-4b9f-9663-8f83a7259587 admin admin] The token cache is being invalidate because project fc39a063842a44b490506d5694449996 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:33:24.855498 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-d4dd34c4-50c9-4b9f-9663-8f83a7259587 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:24.856226 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-d4dd34c4-50c9-4b9f-9663-8f83a7259587 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:24.857883 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 900/1810] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Aug 6 01:33:24 2022] DELETE /identity/v3/projects/fc39a063842a44b490506d5694449996 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:33:24.865453 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1189f0cc-baa4-4a09-a548-23db793293ba None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:24.865673 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1189f0cc-baa4-4a09-a548-23db793293ba None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:24.865953 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1189f0cc-baa4-4a09-a548-23db793293ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:24.892169 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-1189f0cc-baa4-4a09-a548-23db793293ba None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:33:24.894010 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-1189f0cc-baa4-4a09-a548-23db793293ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:24.942728 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 911/1811] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:33:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:33:24.946191 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1840ee63-cf91-4efc-83d4-7620b3c63e6a None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:24.949933 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-1840ee63-cf91-4efc-83d4-7620b3c63e6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:24.976475 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1840ee63-cf91-4efc-83d4-7620b3c63e6a None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:24.981408 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 901/1812] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:33:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:33:24.982329 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1189f0cc-baa4-4a09-a548-23db793293ba None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:24.983932 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-1189f0cc-baa4-4a09-a548-23db793293ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:24.993235 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bace85f2-c4c0-4d12-954d-bc177167f162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:24.993904 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bace85f2-c4c0-4d12-954d-bc177167f162 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:24.994194 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bace85f2-c4c0-4d12-954d-bc177167f162 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:24.994465 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bace85f2-c4c0-4d12-954d-bc177167f162 admin admin] PATH_INFO: `/v3/users/be6ce7c263a14206978a2402d4832a4c` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:24.995687 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bace85f2-c4c0-4d12-954d-bc177167f162 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be6ce7c263a14206978a2402d4832a4c)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:24.997674 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bace85f2-c4c0-4d12-954d-bc177167f162 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:25.041741 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:33:25.075393 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1189f0cc-baa4-4a09-a548-23db793293ba None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:25.080105 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-bace85f2-c4c0-4d12-954d-bc177167f162 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be6ce7c263a14206978a2402d4832a4c'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:25.081200 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78475bee-7811-4f1a-8375-d759e590c0f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:25.081949 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78475bee-7811-4f1a-8375-d759e590c0f7 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:25.082236 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78475bee-7811-4f1a-8375-d759e590c0f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:25.082854 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78475bee-7811-4f1a-8375-d759e590c0f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:25.091076 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-bace85f2-c4c0-4d12-954d-bc177167f162 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be6ce7c263a14206978a2402d4832a4c'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:25.092008 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-bace85f2-c4c0-4d12-954d-bc177167f162 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be6ce7c263a14206978a2402d4832a4c'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:25.092710 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-bace85f2-c4c0-4d12-954d-bc177167f162 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be6ce7c263a14206978a2402d4832a4c'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:25.096765 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78475bee-7811-4f1a-8375-d759e590c0f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:25.099435 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78475bee-7811-4f1a-8375-d759e590c0f7 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:25.111709 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-bace85f2-c4c0-4d12-954d-bc177167f162 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be6ce7c263a14206978a2402d4832a4c'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:25.129526 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 902/1813] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:33:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:33:25.131371 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee757fe6-f0f4-431c-b639-367897593a80 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:25.132395 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee757fe6-f0f4-431c-b639-367897593a80 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:25.132993 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee757fe6-f0f4-431c-b639-367897593a80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:25.160513 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-ee757fe6-f0f4-431c-b639-367897593a80 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:33:25.161827 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-ee757fe6-f0f4-431c-b639-367897593a80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:25.164611 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 912/1814] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Aug 6 01:33:24 2022] DELETE /identity/v3/users/be6ce7c263a14206978a2402d4832a4c => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:33:25.168998 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b37a04d-c17b-43ab-b159-7f8e9e2fb4e3 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:25.169667 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b37a04d-c17b-43ab-b159-7f8e9e2fb4e3 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:25.170148 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b37a04d-c17b-43ab-b159-7f8e9e2fb4e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:25.214157 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-2b37a04d-c17b-43ab-b159-7f8e9e2fb4e3 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:33:25.215408 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-2b37a04d-c17b-43ab-b159-7f8e9e2fb4e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:25.274782 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:33:25.276612 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 903/1815] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:33:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:33:25.281115 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee757fe6-f0f4-431c-b639-367897593a80 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:25.309411 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee757fe6-f0f4-431c-b639-367897593a80 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:25.314246 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 913/1816] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:33:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:33:25.318024 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a13507-f504-490c-8920-ac6f5d4f360c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:25.319147 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a13507-f504-490c-8920-ac6f5d4f360c admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:25.319501 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a13507-f504-490c-8920-ac6f5d4f360c admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:25.319853 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a13507-f504-490c-8920-ac6f5d4f360c admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:25.320696 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a13507-f504-490c-8920-ac6f5d4f360c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:33:25.321266 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a13507-f504-490c-8920-ac6f5d4f360c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:25.322045 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b37a04d-c17b-43ab-b159-7f8e9e2fb4e3 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:25.323474 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a13507-f504-490c-8920-ac6f5d4f360c admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:25.326776 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-2b37a04d-c17b-43ab-b159-7f8e9e2fb4e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:25.340846 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 904/1817] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Aug 6 01:33:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:33:25.346607 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a13507-f504-490c-8920-ac6f5d4f360c admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:25.360784 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a13507-f504-490c-8920-ac6f5d4f360c admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:25.367689 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559d4955-2995-4410-8792-8d9b26c5e0e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:25.368624 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559d4955-2995-4410-8792-8d9b26c5e0e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:25.369149 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559d4955-2995-4410-8792-8d9b26c5e0e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:25.369623 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559d4955-2995-4410-8792-8d9b26c5e0e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:25.370369 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559d4955-2995-4410-8792-8d9b26c5e0e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:25.373017 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559d4955-2995-4410-8792-8d9b26c5e0e5 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:25.422404 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 905/1818] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Aug 6 01:33:25 2022] POST /identity/v3/projects => generated 385 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:33:25.424606 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b6b39e-fe1c-4d77-a70b-de7ec76297a3 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:25.425226 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b6b39e-fe1c-4d77-a70b-de7ec76297a3 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:25.425959 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b6b39e-fe1c-4d77-a70b-de7ec76297a3 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:25.427736 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 906/1819] 10.222.0.26 () {54 vars in 921 bytes} [Sat Aug 6 01:33:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:33:25.428745 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b6b39e-fe1c-4d77-a70b-de7ec76297a3 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:25.439141 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b37a04d-c17b-43ab-b159-7f8e9e2fb4e3 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:25.522595 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ee89c7-4724-4148-adf9-70a9c9ea0c74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:25.523469 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ee89c7-4724-4148-adf9-70a9c9ea0c74 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:25.523727 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ee89c7-4724-4148-adf9-70a9c9ea0c74 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:25.523956 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ee89c7-4724-4148-adf9-70a9c9ea0c74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:25.550635 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b6b39e-fe1c-4d77-a70b-de7ec76297a3 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:25.557039 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4ee6c8-3cd2-4d3d-bc44-81453d8ea5fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:25.557712 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4ee6c8-3cd2-4d3d-bc44-81453d8ea5fc admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:25.557978 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4ee6c8-3cd2-4d3d-bc44-81453d8ea5fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:25.558336 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4ee6c8-3cd2-4d3d-bc44-81453d8ea5fc admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:25.558958 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4ee6c8-3cd2-4d3d-bc44-81453d8ea5fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:25.560929 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4ee6c8-3cd2-4d3d-bc44-81453d8ea5fc admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:25.588780 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 907/1820] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:33:25 2022] POST /identity/v3/users => generated 293 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:33:25.590097 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb11d1d6-e703-44cc-bf7b-11b5f24cd7c6 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:25.590544 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb11d1d6-e703-44cc-bf7b-11b5f24cd7c6 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:25.590833 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb11d1d6-e703-44cc-bf7b-11b5f24cd7c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:25.610733 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ee89c7-4724-4148-adf9-70a9c9ea0c74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:25.612766 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ee89c7-4724-4148-adf9-70a9c9ea0c74 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:25.614688 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-cb11d1d6-e703-44cc-bf7b-11b5f24cd7c6 None None] MFA Rules not processed for user `b87adc1d5bc240e99f95f6595b350be4`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:33:25.615936 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-cb11d1d6-e703-44cc-bf7b-11b5f24cd7c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:25.625575 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 914/1821] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:33:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 01:33:25.626356 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ee89c7-4724-4148-adf9-70a9c9ea0c74 service neutron] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:25.637887 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ee89c7-4724-4148-adf9-70a9c9ea0c74 service neutron] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:25.642899 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86433dd5-2caf-401c-b06a-eb7174f226c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:25.644128 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86433dd5-2caf-401c-b06a-eb7174f226c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:25.644417 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86433dd5-2caf-401c-b06a-eb7174f226c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:25.644696 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86433dd5-2caf-401c-b06a-eb7174f226c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:25.645438 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86433dd5-2caf-401c-b06a-eb7174f226c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:25.647116 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86433dd5-2caf-401c-b06a-eb7174f226c3 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:25.657324 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:33:25.661278 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 915/1822] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:33:25 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:33:25.665327 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86433dd5-2caf-401c-b06a-eb7174f226c3 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:25.679287 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86433dd5-2caf-401c-b06a-eb7174f226c3 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:25.686315 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b022ef2d-012e-43d9-9558-68308217e5a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:25.688849 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b022ef2d-012e-43d9-9558-68308217e5a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:25.688849 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b022ef2d-012e-43d9-9558-68308217e5a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:25.688849 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b022ef2d-012e-43d9-9558-68308217e5a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:25.688849 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b022ef2d-012e-43d9-9558-68308217e5a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:25.690683 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b022ef2d-012e-43d9-9558-68308217e5a8 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:25.704840 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 916/1823] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:33:25 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 01:33:25.710633 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b022ef2d-012e-43d9-9558-68308217e5a8 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:25.720611 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b022ef2d-012e-43d9-9558-68308217e5a8 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:25.720994 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 908/1824] 10.222.0.26 () {58 vars in 1018 bytes} [Sat Aug 6 01:33:25 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:33:25.725713 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90503ded-0fec-418a-a828-2f6f6686ebf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:25.732426 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90503ded-0fec-418a-a828-2f6f6686ebf5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:25.732720 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90503ded-0fec-418a-a828-2f6f6686ebf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:25.733018 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90503ded-0fec-418a-a828-2f6f6686ebf5 admin admin] PATH_INFO: `/v3/projects/2a4663cc11bb4153bf690d36f499ce27/users/821ef914d521425fb2e558c0d8e8e885/roles/61b3b93d37a047e4bb8369b46fe20db9` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:25.733950 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb11d1d6-e703-44cc-bf7b-11b5f24cd7c6 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:25.735753 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-cb11d1d6-e703-44cc-bf7b-11b5f24cd7c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:25.759199 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90503ded-0fec-418a-a828-2f6f6686ebf5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a4663cc11bb4153bf690d36f499ce27, user_id=821ef914d521425fb2e558c0d8e8e885, role_id=61b3b93d37a047e4bb8369b46fe20db9)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:25.760605 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb11d1d6-e703-44cc-bf7b-11b5f24cd7c6 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:25.762148 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90503ded-0fec-418a-a828-2f6f6686ebf5 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:25.776003 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf22fc2f-be18-4eb5-967d-1ef2cfcafa23 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87adc1d5bc240e99f95f6595b350be4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:25.777781 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf22fc2f-be18-4eb5-967d-1ef2cfcafa23 service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:25.778835 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf22fc2f-be18-4eb5-967d-1ef2cfcafa23 service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:25.779355 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf22fc2f-be18-4eb5-967d-1ef2cfcafa23 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:25.796810 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 917/1825] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:33:25 2022] PUT /identity/v3/projects/2a4663cc11bb4153bf690d36f499ce27/users/821ef914d521425fb2e558c0d8e8e885/roles/61b3b93d37a047e4bb8369b46fe20db9 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:33:25.799525 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf7e89c-2668-4ebd-a1c4-2354529a2d1c None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:25.800008 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf7e89c-2668-4ebd-a1c4-2354529a2d1c None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:25.800684 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf7e89c-2668-4ebd-a1c4-2354529a2d1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:25.816655 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf22fc2f-be18-4eb5-967d-1ef2cfcafa23 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:25.820043 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf22fc2f-be18-4eb5-967d-1ef2cfcafa23 service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:25.821670 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:33:25.841882 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 909/1826] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:33:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:33:25.851655 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-bdf7e89c-2668-4ebd-a1c4-2354529a2d1c None None] MFA Rules not processed for user `821ef914d521425fb2e558c0d8e8e885`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:33:25.855596 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-bdf7e89c-2668-4ebd-a1c4-2354529a2d1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:25.962664 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 918/1827] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:33:25 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:33:25.966685 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf22fc2f-be18-4eb5-967d-1ef2cfcafa23 service nova] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:25.981036 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf22fc2f-be18-4eb5-967d-1ef2cfcafa23 service nova] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:26.063373 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb694f99-b8c2-47f9-8f60-f71034f93b24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:26.064142 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb694f99-b8c2-47f9-8f60-f71034f93b24 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:26.066013 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb694f99-b8c2-47f9-8f60-f71034f93b24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:26.066013 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb694f99-b8c2-47f9-8f60-f71034f93b24 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:26.066013 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb694f99-b8c2-47f9-8f60-f71034f93b24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:26.068653 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb694f99-b8c2-47f9-8f60-f71034f93b24 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:26.112501 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 910/1828] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Aug 6 01:33:25 2022] POST /identity/v3/projects => generated 385 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:33:26.116520 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf7e89c-2668-4ebd-a1c4-2354529a2d1c None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:26.136349 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf7e89c-2668-4ebd-a1c4-2354529a2d1c None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:26.211178 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc105d1-0553-4e2c-bead-01c32dbebf70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:26.211805 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc105d1-0553-4e2c-bead-01c32dbebf70 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:26.212085 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc105d1-0553-4e2c-bead-01c32dbebf70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:26.212355 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc105d1-0553-4e2c-bead-01c32dbebf70 admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:26.213017 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc105d1-0553-4e2c-bead-01c32dbebf70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:26.214990 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc105d1-0553-4e2c-bead-01c32dbebf70 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:26.239107 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 919/1829] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:33:26 2022] POST /identity/v3/users => generated 299 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:33:26.242003 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb694f99-b8c2-47f9-8f60-f71034f93b24 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:26.253152 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb694f99-b8c2-47f9-8f60-f71034f93b24 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:26.257815 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6183fe89-cb04-454d-bb28-108b9dedf2cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:26.258432 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6183fe89-cb04-454d-bb28-108b9dedf2cc admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:26.258714 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6183fe89-cb04-454d-bb28-108b9dedf2cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:26.258985 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6183fe89-cb04-454d-bb28-108b9dedf2cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:26.259669 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6183fe89-cb04-454d-bb28-108b9dedf2cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:26.261290 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6183fe89-cb04-454d-bb28-108b9dedf2cc admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:26.273173 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 911/1830] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:33:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:33:26.277651 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc105d1-0553-4e2c-bead-01c32dbebf70 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:26.288691 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc105d1-0553-4e2c-bead-01c32dbebf70 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:26.293426 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d376bf17-0b23-4807-afe7-959f5fb38474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:26.294093 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d376bf17-0b23-4807-afe7-959f5fb38474 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:26.294431 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d376bf17-0b23-4807-afe7-959f5fb38474 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:26.294710 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d376bf17-0b23-4807-afe7-959f5fb38474 admin admin] PATH_INFO: `/v3/projects/a1a230f6ebed4710b351ba5d90821aaa/users/c3eab80a79134e198ae472c31fa1764e/roles/4559b9a0837249f895a13c70096f8d38` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:26.311169 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d376bf17-0b23-4807-afe7-959f5fb38474 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1a230f6ebed4710b351ba5d90821aaa, user_id=c3eab80a79134e198ae472c31fa1764e, role_id=4559b9a0837249f895a13c70096f8d38)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:26.313963 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d376bf17-0b23-4807-afe7-959f5fb38474 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:26.327736 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 920/1831] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:33:26 2022] PUT /identity/v3/projects/a1a230f6ebed4710b351ba5d90821aaa/users/c3eab80a79134e198ae472c31fa1764e/roles/4559b9a0837249f895a13c70096f8d38 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:33:26.332852 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6183fe89-cb04-454d-bb28-108b9dedf2cc admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:26.342531 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6183fe89-cb04-454d-bb28-108b9dedf2cc admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:26.409884 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4caf1e-3264-41d8-b47d-a56d6802f5c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:26.411409 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4caf1e-3264-41d8-b47d-a56d6802f5c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:26.411586 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4caf1e-3264-41d8-b47d-a56d6802f5c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:26.411867 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4caf1e-3264-41d8-b47d-a56d6802f5c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:26.412579 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4caf1e-3264-41d8-b47d-a56d6802f5c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:26.414368 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4caf1e-3264-41d8-b47d-a56d6802f5c2 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:26.431237 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 912/1832] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:33:26 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:33:26.432368 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d376bf17-0b23-4807-afe7-959f5fb38474 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:26.443699 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d376bf17-0b23-4807-afe7-959f5fb38474 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:26.448697 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c12f67e-0111-4315-9429-87be7e2dae40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:26.449467 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c12f67e-0111-4315-9429-87be7e2dae40 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:26.449751 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c12f67e-0111-4315-9429-87be7e2dae40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:26.450339 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c12f67e-0111-4315-9429-87be7e2dae40 admin admin] PATH_INFO: `/v3/projects/a1a230f6ebed4710b351ba5d90821aaa/users/c3eab80a79134e198ae472c31fa1764e/roles/4559b9a0837249f895a13c70096f8d38` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:26.452260 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c12f67e-0111-4315-9429-87be7e2dae40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1a230f6ebed4710b351ba5d90821aaa, user_id=c3eab80a79134e198ae472c31fa1764e, role_id=4559b9a0837249f895a13c70096f8d38)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:26.455024 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c12f67e-0111-4315-9429-87be7e2dae40 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:26.478989 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 921/1833] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:33:26 2022] PUT /identity/v3/projects/a1a230f6ebed4710b351ba5d90821aaa/users/c3eab80a79134e198ae472c31fa1764e/roles/4559b9a0837249f895a13c70096f8d38 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:33:26.481408 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbe006c-38f9-4e0d-b13d-173d9092d5d0 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:26.481704 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbe006c-38f9-4e0d-b13d-173d9092d5d0 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:26.481990 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbe006c-38f9-4e0d-b13d-173d9092d5d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:26.530078 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-3cbe006c-38f9-4e0d-b13d-173d9092d5d0 None None] MFA Rules not processed for user `c3eab80a79134e198ae472c31fa1764e`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:33:26.531703 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-3cbe006c-38f9-4e0d-b13d-173d9092d5d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:26.558669 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:33:26.592102 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c12f67e-0111-4315-9429-87be7e2dae40 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:26.593951 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-9c12f67e-0111-4315-9429-87be7e2dae40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:26.654110 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 913/1834] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:33:26 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:33:26.667341 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbe006c-38f9-4e0d-b13d-173d9092d5d0 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:26.670113 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-3cbe006c-38f9-4e0d-b13d-173d9092d5d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:26.698665 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:33:26.720482 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c12f67e-0111-4315-9429-87be7e2dae40 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:26.726105 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd69032f-560d-4cc2-82bb-75bc24f10123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:26.727140 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd69032f-560d-4cc2-82bb-75bc24f10123 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:26.727347 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd69032f-560d-4cc2-82bb-75bc24f10123 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:26.727588 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd69032f-560d-4cc2-82bb-75bc24f10123 admin admin] PATH_INFO: `/v3/projects/e854c9621e3947a98095554cdcd8c3ba` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:26.729061 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd69032f-560d-4cc2-82bb-75bc24f10123 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e854c9621e3947a98095554cdcd8c3ba)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:26.730982 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd69032f-560d-4cc2-82bb-75bc24f10123 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:26.770866 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbe006c-38f9-4e0d-b13d-173d9092d5d0 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:26.774758 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42ab5f8-c616-44ec-a881-2d5800e8db0a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1be1e6173e374a60b9e1fbf51a1ab571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:26.775430 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42ab5f8-c616-44ec-a881-2d5800e8db0a service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:26.776194 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42ab5f8-c616-44ec-a881-2d5800e8db0a service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:26.776714 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42ab5f8-c616-44ec-a881-2d5800e8db0a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:26.798018 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-bd69032f-560d-4cc2-82bb-75bc24f10123 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e854c9621e3947a98095554cdcd8c3ba', 'request_id': 'req-bd69032f-560d-4cc2-82bb-75bc24f10123'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:26.807994 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-bd69032f-560d-4cc2-82bb-75bc24f10123 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e854c9621e3947a98095554cdcd8c3ba', 'request_id': 'req-bd69032f-560d-4cc2-82bb-75bc24f10123'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:26.816411 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-bd69032f-560d-4cc2-82bb-75bc24f10123 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e854c9621e3947a98095554cdcd8c3ba', 'request_id': 'req-bd69032f-560d-4cc2-82bb-75bc24f10123'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:26.843465 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-bd69032f-560d-4cc2-82bb-75bc24f10123 admin admin] The token cache is being invalidate because project e854c9621e3947a98095554cdcd8c3ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:33:26.844505 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-bd69032f-560d-4cc2-82bb-75bc24f10123 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:26.845641 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-bd69032f-560d-4cc2-82bb-75bc24f10123 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:26.850837 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 922/1835] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Aug 6 01:33:26 2022] DELETE /identity/v3/projects/e854c9621e3947a98095554cdcd8c3ba => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:33:26.854230 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd69032f-560d-4cc2-82bb-75bc24f10123 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:26.857497 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-bd69032f-560d-4cc2-82bb-75bc24f10123 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:26.867856 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42ab5f8-c616-44ec-a881-2d5800e8db0a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:26.869401 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42ab5f8-c616-44ec-a881-2d5800e8db0a service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:26.903259 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 914/1836] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:33:26 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:33:26.957645 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46174d3-2fcd-4850-80f7-fed519a61c3d None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:26.957645 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46174d3-2fcd-4850-80f7-fed519a61c3d None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:26.957892 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46174d3-2fcd-4850-80f7-fed519a61c3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:26.991736 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd69032f-560d-4cc2-82bb-75bc24f10123 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:26.997631 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2d8c42-3aa6-4f67-9c0c-bb0d8fe052ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:26.998394 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2d8c42-3aa6-4f67-9c0c-bb0d8fe052ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:26.999050 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2d8c42-3aa6-4f67-9c0c-bb0d8fe052ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:26.999525 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2d8c42-3aa6-4f67-9c0c-bb0d8fe052ca admin admin] PATH_INFO: `/v3/users/ad7d276321c54bf993f6d7c4ff3a6ef5` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:27.000874 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2d8c42-3aa6-4f67-9c0c-bb0d8fe052ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad7d276321c54bf993f6d7c4ff3a6ef5)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:27.003908 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2d8c42-3aa6-4f67-9c0c-bb0d8fe052ca admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:27.004513 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-b46174d3-2fcd-4850-80f7-fed519a61c3d None None] MFA Rules not processed for user `3f081e5336f74e21903f11df0b9d95b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:33:27.006273 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-b46174d3-2fcd-4850-80f7-fed519a61c3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:27.087002 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:33:27.089721 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-8e2d8c42-3aa6-4f67-9c0c-bb0d8fe052ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad7d276321c54bf993f6d7c4ff3a6ef5'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:27.099106 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-8e2d8c42-3aa6-4f67-9c0c-bb0d8fe052ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad7d276321c54bf993f6d7c4ff3a6ef5'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:27.099927 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-8e2d8c42-3aa6-4f67-9c0c-bb0d8fe052ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad7d276321c54bf993f6d7c4ff3a6ef5'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:27.100640 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-8e2d8c42-3aa6-4f67-9c0c-bb0d8fe052ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad7d276321c54bf993f6d7c4ff3a6ef5'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:27.115066 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 915/1837] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:33:26 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:33:27.121692 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-8e2d8c42-3aa6-4f67-9c0c-bb0d8fe052ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad7d276321c54bf993f6d7c4ff3a6ef5'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:27.126163 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46174d3-2fcd-4850-80f7-fed519a61c3d None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:27.136732 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-b46174d3-2fcd-4850-80f7-fed519a61c3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:27.137406 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:33:27.150287 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 923/1838] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Aug 6 01:33:26 2022] DELETE /identity/v3/users/ad7d276321c54bf993f6d7c4ff3a6ef5 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:33:27.224865 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46174d3-2fcd-4850-80f7-fed519a61c3d None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:27.307792 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75cd5de-a8b5-4aa4-88e4-95d12e1b6f52 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1be1e6173e374a60b9e1fbf51a1ab571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:27.308519 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75cd5de-a8b5-4aa4-88e4-95d12e1b6f52 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:27.308871 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75cd5de-a8b5-4aa4-88e4-95d12e1b6f52 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:27.309244 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75cd5de-a8b5-4aa4-88e4-95d12e1b6f52 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:27.311381 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-c75cd5de-a8b5-4aa4-88e4-95d12e1b6f52 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:27.432893 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75cd5de-a8b5-4aa4-88e4-95d12e1b6f52 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:27.435404 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75cd5de-a8b5-4aa4-88e4-95d12e1b6f52 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:27.446966 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 916/1839] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:33:27 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:33:27.561933 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2d8c42-3aa6-4f67-9c0c-bb0d8fe052ca admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:27.563564 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-8e2d8c42-3aa6-4f67-9c0c-bb0d8fe052ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:27.691146 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2d8c42-3aa6-4f67-9c0c-bb0d8fe052ca admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:27.699156 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450923d2-2886-486e-b047-53ad8631a085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:27.700420 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450923d2-2886-486e-b047-53ad8631a085 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:27.700936 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450923d2-2886-486e-b047-53ad8631a085 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:27.701475 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450923d2-2886-486e-b047-53ad8631a085 admin admin] PATH_INFO: `/v3/projects/1ead2b3e0775428eadabddb4256c5b3d` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:27.703523 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450923d2-2886-486e-b047-53ad8631a085 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ead2b3e0775428eadabddb4256c5b3d)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:27.708148 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450923d2-2886-486e-b047-53ad8631a085 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:27.790815 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-450923d2-2886-486e-b047-53ad8631a085 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ead2b3e0775428eadabddb4256c5b3d', 'request_id': 'req-450923d2-2886-486e-b047-53ad8631a085'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:27.803886 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-450923d2-2886-486e-b047-53ad8631a085 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ead2b3e0775428eadabddb4256c5b3d', 'request_id': 'req-450923d2-2886-486e-b047-53ad8631a085'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:27.819694 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-450923d2-2886-486e-b047-53ad8631a085 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ead2b3e0775428eadabddb4256c5b3d', 'request_id': 'req-450923d2-2886-486e-b047-53ad8631a085'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:27.853189 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-450923d2-2886-486e-b047-53ad8631a085 admin admin] The token cache is being invalidate because project 1ead2b3e0775428eadabddb4256c5b3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:33:27.853930 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-450923d2-2886-486e-b047-53ad8631a085 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:27.855630 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-450923d2-2886-486e-b047-53ad8631a085 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:27.859345 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 924/1840] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Aug 6 01:33:27 2022] DELETE /identity/v3/projects/1ead2b3e0775428eadabddb4256c5b3d => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:33:27.865675 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786a87aa-353b-41da-9294-1e102cf88676 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:27.866157 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786a87aa-353b-41da-9294-1e102cf88676 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:27.866610 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786a87aa-353b-41da-9294-1e102cf88676 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:27.897279 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-786a87aa-353b-41da-9294-1e102cf88676 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:33:27.899978 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-786a87aa-353b-41da-9294-1e102cf88676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:27.971592 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 917/1841] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:33:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:33:27.979323 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da01e3aa-a3c7-43b4-b7f3-d2980162a093 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:27.979609 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da01e3aa-a3c7-43b4-b7f3-d2980162a093 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:27.979938 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da01e3aa-a3c7-43b4-b7f3-d2980162a093 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:28.000099 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-da01e3aa-a3c7-43b4-b7f3-d2980162a093 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:33:28.001344 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-da01e3aa-a3c7-43b4-b7f3-d2980162a093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:28.010214 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 925/1842] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:33:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:33:28.014511 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786a87aa-353b-41da-9294-1e102cf88676 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:28.020215 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:33:28.025602 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786a87aa-353b-41da-9294-1e102cf88676 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:28.030908 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcbd6fc-5fd7-4b0e-b832-eb491f6aa177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:28.031525 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcbd6fc-5fd7-4b0e-b832-eb491f6aa177 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:28.031805 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcbd6fc-5fd7-4b0e-b832-eb491f6aa177 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:28.032200 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcbd6fc-5fd7-4b0e-b832-eb491f6aa177 admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:28.032905 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcbd6fc-5fd7-4b0e-b832-eb491f6aa177 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:33:28.033294 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcbd6fc-5fd7-4b0e-b832-eb491f6aa177 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:28.035251 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcbd6fc-5fd7-4b0e-b832-eb491f6aa177 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:28.047975 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 918/1843] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Aug 6 01:33:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:33:28.052715 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da01e3aa-a3c7-43b4-b7f3-d2980162a093 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:28.061947 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da01e3aa-a3c7-43b4-b7f3-d2980162a093 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:28.079145 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab0862d-2845-431c-ba10-8b69c27b8ab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:28.079925 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab0862d-2845-431c-ba10-8b69c27b8ab3 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:28.080228 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab0862d-2845-431c-ba10-8b69c27b8ab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:28.081158 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab0862d-2845-431c-ba10-8b69c27b8ab3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:28.081851 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab0862d-2845-431c-ba10-8b69c27b8ab3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:28.083974 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab0862d-2845-431c-ba10-8b69c27b8ab3 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:28.119141 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 926/1844] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Aug 6 01:33:28 2022] POST /identity/v3/projects => generated 381 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:33:28.123398 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcbd6fc-5fd7-4b0e-b832-eb491f6aa177 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:28.133968 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcbd6fc-5fd7-4b0e-b832-eb491f6aa177 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:28.192083 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca757359-859d-472e-a27b-95289e8f5b54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:28.192720 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca757359-859d-472e-a27b-95289e8f5b54 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:28.193006 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca757359-859d-472e-a27b-95289e8f5b54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:28.193325 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca757359-859d-472e-a27b-95289e8f5b54 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:28.194024 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca757359-859d-472e-a27b-95289e8f5b54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:28.195960 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca757359-859d-472e-a27b-95289e8f5b54 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:28.220689 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 919/1845] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:33:28 2022] POST /identity/v3/users => generated 291 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:33:28.222071 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab0862d-2845-431c-ba10-8b69c27b8ab3 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:28.232223 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab0862d-2845-431c-ba10-8b69c27b8ab3 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:28.237498 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d04ee8-257f-41e5-8042-5d5c23914fea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:28.238111 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d04ee8-257f-41e5-8042-5d5c23914fea admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:28.238387 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d04ee8-257f-41e5-8042-5d5c23914fea admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:28.238657 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d04ee8-257f-41e5-8042-5d5c23914fea admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:28.239632 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d04ee8-257f-41e5-8042-5d5c23914fea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:28.241241 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d04ee8-257f-41e5-8042-5d5c23914fea admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:28.251489 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 927/1846] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:33:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:33:28.256311 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca757359-859d-472e-a27b-95289e8f5b54 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:28.266400 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca757359-859d-472e-a27b-95289e8f5b54 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:28.271218 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fede0f87-4afe-4993-a523-58b6a9fa2358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:28.272167 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fede0f87-4afe-4993-a523-58b6a9fa2358 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:28.272167 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fede0f87-4afe-4993-a523-58b6a9fa2358 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:28.272500 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fede0f87-4afe-4993-a523-58b6a9fa2358 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:28.273157 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fede0f87-4afe-4993-a523-58b6a9fa2358 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:28.274712 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fede0f87-4afe-4993-a523-58b6a9fa2358 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:28.289166 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 920/1847] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:33:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:33:28.291450 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d04ee8-257f-41e5-8042-5d5c23914fea admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:28.309221 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d04ee8-257f-41e5-8042-5d5c23914fea admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:28.313884 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d676b36e-f3f5-42fd-8bd6-629b83e192d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:28.314550 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d676b36e-f3f5-42fd-8bd6-629b83e192d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:28.314925 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d676b36e-f3f5-42fd-8bd6-629b83e192d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:28.315223 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d676b36e-f3f5-42fd-8bd6-629b83e192d4 admin admin] PATH_INFO: `/v3/projects/8bfb1d46de784327ae9a18cc5305f1da/users/0e22e55ca1e24bdaaf6c875d2625981d/roles/61b3b93d37a047e4bb8369b46fe20db9` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:28.329517 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d676b36e-f3f5-42fd-8bd6-629b83e192d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bfb1d46de784327ae9a18cc5305f1da, user_id=0e22e55ca1e24bdaaf6c875d2625981d, role_id=61b3b93d37a047e4bb8369b46fe20db9)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:28.332817 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d676b36e-f3f5-42fd-8bd6-629b83e192d4 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:28.346776 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 928/1848] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:33:28 2022] PUT /identity/v3/projects/8bfb1d46de784327ae9a18cc5305f1da/users/0e22e55ca1e24bdaaf6c875d2625981d/roles/61b3b93d37a047e4bb8369b46fe20db9 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:33:28.357435 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dcb556-c527-4207-9132-c6251f27285b None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:28.357923 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dcb556-c527-4207-9132-c6251f27285b None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:28.358022 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dcb556-c527-4207-9132-c6251f27285b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:28.385416 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-33dcb556-c527-4207-9132-c6251f27285b None None] MFA Rules not processed for user `0e22e55ca1e24bdaaf6c875d2625981d`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:33:28.386633 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-33dcb556-c527-4207-9132-c6251f27285b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:28.454252 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 921/1849] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:33:28 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:33:28.459373 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d676b36e-f3f5-42fd-8bd6-629b83e192d4 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:28.486263 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:33:28.523859 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d676b36e-f3f5-42fd-8bd6-629b83e192d4 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:28.528596 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85b7436-16f7-42c2-a594-be6fc49df0ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:28.529252 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85b7436-16f7-42c2-a594-be6fc49df0ed admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:28.529535 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85b7436-16f7-42c2-a594-be6fc49df0ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:28.529811 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85b7436-16f7-42c2-a594-be6fc49df0ed admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:28.530479 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85b7436-16f7-42c2-a594-be6fc49df0ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:28.532456 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85b7436-16f7-42c2-a594-be6fc49df0ed admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:28.560576 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 929/1850] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Aug 6 01:33:28 2022] POST /identity/v3/projects => generated 381 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:33:28.564953 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dcb556-c527-4207-9132-c6251f27285b None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:28.575579 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dcb556-c527-4207-9132-c6251f27285b None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:28.625643 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:33:28.633003 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555dbafd-af1d-49bc-9fa0-c17a89142d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:28.633663 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555dbafd-af1d-49bc-9fa0-c17a89142d72 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:28.634141 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555dbafd-af1d-49bc-9fa0-c17a89142d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:28.634626 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555dbafd-af1d-49bc-9fa0-c17a89142d72 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:28.635329 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555dbafd-af1d-49bc-9fa0-c17a89142d72 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:28.637341 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555dbafd-af1d-49bc-9fa0-c17a89142d72 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:28.659312 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 922/1851] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:33:28 2022] POST /identity/v3/users => generated 297 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:33:28.663351 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85b7436-16f7-42c2-a594-be6fc49df0ed admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:28.673548 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85b7436-16f7-42c2-a594-be6fc49df0ed admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:28.678794 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d16481-7431-44f2-804d-7696ec1e4fed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:28.679400 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d16481-7431-44f2-804d-7696ec1e4fed admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:28.679681 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d16481-7431-44f2-804d-7696ec1e4fed admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:28.679957 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d16481-7431-44f2-804d-7696ec1e4fed admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:28.680648 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d16481-7431-44f2-804d-7696ec1e4fed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:28.682282 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d16481-7431-44f2-804d-7696ec1e4fed admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:28.693008 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 930/1852] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:33:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:33:28.697167 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555dbafd-af1d-49bc-9fa0-c17a89142d72 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:28.707044 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555dbafd-af1d-49bc-9fa0-c17a89142d72 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:28.711655 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfc512e-6e97-4b73-898b-58747cb325b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:28.712319 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfc512e-6e97-4b73-898b-58747cb325b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:28.712605 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfc512e-6e97-4b73-898b-58747cb325b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:28.712890 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfc512e-6e97-4b73-898b-58747cb325b6 admin admin] PATH_INFO: `/v3/projects/e1bb0f8e5e2f43ee92b7c00fa2f029ab/users/7929a339bb034023816487487ba93886/roles/4559b9a0837249f895a13c70096f8d38` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:28.729100 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfc512e-6e97-4b73-898b-58747cb325b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1bb0f8e5e2f43ee92b7c00fa2f029ab, user_id=7929a339bb034023816487487ba93886, role_id=4559b9a0837249f895a13c70096f8d38)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:28.731799 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfc512e-6e97-4b73-898b-58747cb325b6 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:28.745149 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 923/1853] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:33:28 2022] PUT /identity/v3/projects/e1bb0f8e5e2f43ee92b7c00fa2f029ab/users/7929a339bb034023816487487ba93886/roles/4559b9a0837249f895a13c70096f8d38 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:33:28.749029 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d16481-7431-44f2-804d-7696ec1e4fed admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:28.759357 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d16481-7431-44f2-804d-7696ec1e4fed admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:28.815781 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193ccfb6-51b8-4410-977b-cf563aa25d62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:28.816391 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193ccfb6-51b8-4410-977b-cf563aa25d62 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:28.816675 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193ccfb6-51b8-4410-977b-cf563aa25d62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:28.816955 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193ccfb6-51b8-4410-977b-cf563aa25d62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:28.817677 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193ccfb6-51b8-4410-977b-cf563aa25d62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:28.819274 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193ccfb6-51b8-4410-977b-cf563aa25d62 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:28.828731 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 931/1854] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:33:28 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:33:28.833105 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfc512e-6e97-4b73-898b-58747cb325b6 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:28.843176 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfc512e-6e97-4b73-898b-58747cb325b6 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:28.848423 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4e1a84-e2af-44ff-89af-4dfeada2b808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:28.849117 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4e1a84-e2af-44ff-89af-4dfeada2b808 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:28.849406 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4e1a84-e2af-44ff-89af-4dfeada2b808 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:28.849687 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4e1a84-e2af-44ff-89af-4dfeada2b808 admin admin] PATH_INFO: `/v3/projects/e1bb0f8e5e2f43ee92b7c00fa2f029ab/users/7929a339bb034023816487487ba93886/roles/4559b9a0837249f895a13c70096f8d38` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:28.851637 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4e1a84-e2af-44ff-89af-4dfeada2b808 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1bb0f8e5e2f43ee92b7c00fa2f029ab, user_id=7929a339bb034023816487487ba93886, role_id=4559b9a0837249f895a13c70096f8d38)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:28.854318 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4e1a84-e2af-44ff-89af-4dfeada2b808 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:28.865462 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 924/1855] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:33:28 2022] PUT /identity/v3/projects/e1bb0f8e5e2f43ee92b7c00fa2f029ab/users/7929a339bb034023816487487ba93886/roles/4559b9a0837249f895a13c70096f8d38 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:33:28.876369 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac58c74d-d5a9-4196-8327-ef94c60b9a32 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:28.876660 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac58c74d-d5a9-4196-8327-ef94c60b9a32 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:28.876936 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac58c74d-d5a9-4196-8327-ef94c60b9a32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:28.904288 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-ac58c74d-d5a9-4196-8327-ef94c60b9a32 None None] MFA Rules not processed for user `7929a339bb034023816487487ba93886`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:33:28.905549 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-ac58c74d-d5a9-4196-8327-ef94c60b9a32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:28.974804 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 932/1856] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:33:28 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:33:28.987725 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4e1a84-e2af-44ff-89af-4dfeada2b808 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:28.989706 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-fd4e1a84-e2af-44ff-89af-4dfeada2b808 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:29.045743 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4e1a84-e2af-44ff-89af-4dfeada2b808 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:29.049567 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50022bf1-1505-4ebf-9fb5-9054a9bdc002 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1be1e6173e374a60b9e1fbf51a1ab571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:29.050178 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50022bf1-1505-4ebf-9fb5-9054a9bdc002 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:29.050459 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50022bf1-1505-4ebf-9fb5-9054a9bdc002 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:29.050736 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50022bf1-1505-4ebf-9fb5-9054a9bdc002 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:29.059546 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50022bf1-1505-4ebf-9fb5-9054a9bdc002 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:29.060963 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50022bf1-1505-4ebf-9fb5-9054a9bdc002 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:29.070593 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 925/1857] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:33:28 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:33:29.493242 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac58c74d-d5a9-4196-8327-ef94c60b9a32 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:29.512792 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac58c74d-d5a9-4196-8327-ef94c60b9a32 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:29.519349 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fd0a13-ee10-47ba-9277-b44f75937922 tempest-VolumeQuotaClassesTest-1500889916 tempest-VolumeQuotaClassesTest-1500889916-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7929a339bb034023816487487ba93886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1bb0f8e5e2f43ee92b7c00fa2f029ab', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:29.520120 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fd0a13-ee10-47ba-9277-b44f75937922 tempest-VolumeQuotaClassesTest-1500889916 tempest-VolumeQuotaClassesTest-1500889916-project-admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:29.520525 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fd0a13-ee10-47ba-9277-b44f75937922 tempest-VolumeQuotaClassesTest-1500889916 tempest-VolumeQuotaClassesTest-1500889916-project-admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:29.520897 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fd0a13-ee10-47ba-9277-b44f75937922 tempest-VolumeQuotaClassesTest-1500889916 tempest-VolumeQuotaClassesTest-1500889916-project-admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:29.521719 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fd0a13-ee10-47ba-9277-b44f75937922 tempest-VolumeQuotaClassesTest-1500889916 tempest-VolumeQuotaClassesTest-1500889916-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:33:29.522040 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fd0a13-ee10-47ba-9277-b44f75937922 tempest-VolumeQuotaClassesTest-1500889916 tempest-VolumeQuotaClassesTest-1500889916-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:29.524160 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fd0a13-ee10-47ba-9277-b44f75937922 tempest-VolumeQuotaClassesTest-1500889916 tempest-VolumeQuotaClassesTest-1500889916-project-admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:29.540678 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 933/1858] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Aug 6 01:33:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:33:29.542043 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50022bf1-1505-4ebf-9fb5-9054a9bdc002 service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:29.555183 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50022bf1-1505-4ebf-9fb5-9054a9bdc002 service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:29.561095 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e68227-9f1f-4e60-9bf8-3e54c4bc73e6 tempest-VolumeQuotaClassesTest-1500889916 tempest-VolumeQuotaClassesTest-1500889916-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7929a339bb034023816487487ba93886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1bb0f8e5e2f43ee92b7c00fa2f029ab', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:29.561957 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e68227-9f1f-4e60-9bf8-3e54c4bc73e6 tempest-VolumeQuotaClassesTest-1500889916 tempest-VolumeQuotaClassesTest-1500889916-project-admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:29.562317 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e68227-9f1f-4e60-9bf8-3e54c4bc73e6 tempest-VolumeQuotaClassesTest-1500889916 tempest-VolumeQuotaClassesTest-1500889916-project-admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:29.562697 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e68227-9f1f-4e60-9bf8-3e54c4bc73e6 tempest-VolumeQuotaClassesTest-1500889916 tempest-VolumeQuotaClassesTest-1500889916-project-admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:29.563503 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e68227-9f1f-4e60-9bf8-3e54c4bc73e6 tempest-VolumeQuotaClassesTest-1500889916 tempest-VolumeQuotaClassesTest-1500889916-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:29.566132 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e68227-9f1f-4e60-9bf8-3e54c4bc73e6 tempest-VolumeQuotaClassesTest-1500889916 tempest-VolumeQuotaClassesTest-1500889916-project-admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:29.613165 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 926/1859] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Aug 6 01:33:29 2022] POST /identity/v3/projects => generated 354 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:33:29.617585 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fd0a13-ee10-47ba-9277-b44f75937922 tempest-VolumeQuotaClassesTest-1500889916 tempest-VolumeQuotaClassesTest-1500889916-project-admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:29.634196 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fd0a13-ee10-47ba-9277-b44f75937922 tempest-VolumeQuotaClassesTest-1500889916 tempest-VolumeQuotaClassesTest-1500889916-project-admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:29.638473 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:33:29.694580 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40cdeea-172b-42e7-9915-5790aa2bcfd9 tempest-VolumeQuotaClassesTest-1500889916 tempest-VolumeQuotaClassesTest-1500889916-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7929a339bb034023816487487ba93886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1bb0f8e5e2f43ee92b7c00fa2f029ab', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:29.695231 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40cdeea-172b-42e7-9915-5790aa2bcfd9 tempest-VolumeQuotaClassesTest-1500889916 tempest-VolumeQuotaClassesTest-1500889916-project-admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:29.695518 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40cdeea-172b-42e7-9915-5790aa2bcfd9 tempest-VolumeQuotaClassesTest-1500889916 tempest-VolumeQuotaClassesTest-1500889916-project-admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:29.695799 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40cdeea-172b-42e7-9915-5790aa2bcfd9 tempest-VolumeQuotaClassesTest-1500889916 tempest-VolumeQuotaClassesTest-1500889916-project-admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:29.696493 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40cdeea-172b-42e7-9915-5790aa2bcfd9 tempest-VolumeQuotaClassesTest-1500889916 tempest-VolumeQuotaClassesTest-1500889916-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:33:29.696799 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40cdeea-172b-42e7-9915-5790aa2bcfd9 tempest-VolumeQuotaClassesTest-1500889916 tempest-VolumeQuotaClassesTest-1500889916-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:29.698445 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40cdeea-172b-42e7-9915-5790aa2bcfd9 tempest-VolumeQuotaClassesTest-1500889916 tempest-VolumeQuotaClassesTest-1500889916-project-admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:29.710563 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 934/1860] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Aug 6 01:33:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:33:29.749984 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e68227-9f1f-4e60-9bf8-3e54c4bc73e6 tempest-VolumeQuotaClassesTest-1500889916 tempest-VolumeQuotaClassesTest-1500889916-project-admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:29.760860 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e68227-9f1f-4e60-9bf8-3e54c4bc73e6 tempest-VolumeQuotaClassesTest-1500889916 tempest-VolumeQuotaClassesTest-1500889916-project-admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:29.765483 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa44a56-4d84-4ce8-aa3c-b8a4e3b4a637 tempest-VolumeQuotaClassesTest-1500889916 tempest-VolumeQuotaClassesTest-1500889916-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7929a339bb034023816487487ba93886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1bb0f8e5e2f43ee92b7c00fa2f029ab', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:29.766147 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa44a56-4d84-4ce8-aa3c-b8a4e3b4a637 tempest-VolumeQuotaClassesTest-1500889916 tempest-VolumeQuotaClassesTest-1500889916-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:29.766434 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa44a56-4d84-4ce8-aa3c-b8a4e3b4a637 tempest-VolumeQuotaClassesTest-1500889916 tempest-VolumeQuotaClassesTest-1500889916-project-admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:29.766739 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa44a56-4d84-4ce8-aa3c-b8a4e3b4a637 tempest-VolumeQuotaClassesTest-1500889916 tempest-VolumeQuotaClassesTest-1500889916-project-admin] PATH_INFO: `/v3/projects/c5f00a5bf8f548b7aead8b222aa8d27f` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:29.767919 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa44a56-4d84-4ce8-aa3c-b8a4e3b4a637 tempest-VolumeQuotaClassesTest-1500889916 tempest-VolumeQuotaClassesTest-1500889916-project-admin] RBAC: Authorizing `identity:delete_project(project_id=c5f00a5bf8f548b7aead8b222aa8d27f)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:29.769856 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa44a56-4d84-4ce8-aa3c-b8a4e3b4a637 tempest-VolumeQuotaClassesTest-1500889916 tempest-VolumeQuotaClassesTest-1500889916-project-admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:29.816877 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-1aa44a56-4d84-4ce8-aa3c-b8a4e3b4a637 tempest-VolumeQuotaClassesTest-1500889916 tempest-VolumeQuotaClassesTest-1500889916-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5f00a5bf8f548b7aead8b222aa8d27f', 'request_id': 'req-1aa44a56-4d84-4ce8-aa3c-b8a4e3b4a637'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:29.823596 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-1aa44a56-4d84-4ce8-aa3c-b8a4e3b4a637 tempest-VolumeQuotaClassesTest-1500889916 tempest-VolumeQuotaClassesTest-1500889916-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5f00a5bf8f548b7aead8b222aa8d27f', 'request_id': 'req-1aa44a56-4d84-4ce8-aa3c-b8a4e3b4a637'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:29.829271 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-1aa44a56-4d84-4ce8-aa3c-b8a4e3b4a637 tempest-VolumeQuotaClassesTest-1500889916 tempest-VolumeQuotaClassesTest-1500889916-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5f00a5bf8f548b7aead8b222aa8d27f', 'request_id': 'req-1aa44a56-4d84-4ce8-aa3c-b8a4e3b4a637'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:29.841304 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-1aa44a56-4d84-4ce8-aa3c-b8a4e3b4a637 tempest-VolumeQuotaClassesTest-1500889916 tempest-VolumeQuotaClassesTest-1500889916-project-admin] The token cache is being invalidate because project c5f00a5bf8f548b7aead8b222aa8d27f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:33:29.841591 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-1aa44a56-4d84-4ce8-aa3c-b8a4e3b4a637 tempest-VolumeQuotaClassesTest-1500889916 tempest-VolumeQuotaClassesTest-1500889916-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:29.842219 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-1aa44a56-4d84-4ce8-aa3c-b8a4e3b4a637 tempest-VolumeQuotaClassesTest-1500889916 tempest-VolumeQuotaClassesTest-1500889916-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:29.843719 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 927/1861] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Aug 6 01:33:29 2022] DELETE /identity/v3/projects/c5f00a5bf8f548b7aead8b222aa8d27f => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:33:30.033822 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40cdeea-172b-42e7-9915-5790aa2bcfd9 tempest-VolumeQuotaClassesTest-1500889916 tempest-VolumeQuotaClassesTest-1500889916-project-admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:30.035556 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-b40cdeea-172b-42e7-9915-5790aa2bcfd9 tempest-VolumeQuotaClassesTest-1500889916 tempest-VolumeQuotaClassesTest-1500889916-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:30.100780 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40cdeea-172b-42e7-9915-5790aa2bcfd9 tempest-VolumeQuotaClassesTest-1500889916 tempest-VolumeQuotaClassesTest-1500889916-project-admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:30.157333 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ce03df-56fe-40bd-bcc4-08af37353ba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:30.158002 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ce03df-56fe-40bd-bcc4-08af37353ba6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:30.158284 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ce03df-56fe-40bd-bcc4-08af37353ba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:30.159076 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ce03df-56fe-40bd-bcc4-08af37353ba6 admin admin] PATH_INFO: `/v3/users/0e22e55ca1e24bdaaf6c875d2625981d` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:30.160273 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ce03df-56fe-40bd-bcc4-08af37353ba6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e22e55ca1e24bdaaf6c875d2625981d)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:30.162217 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ce03df-56fe-40bd-bcc4-08af37353ba6 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:30.207329 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-30ce03df-56fe-40bd-bcc4-08af37353ba6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e22e55ca1e24bdaaf6c875d2625981d'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:30.209304 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:33:30.215270 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-30ce03df-56fe-40bd-bcc4-08af37353ba6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e22e55ca1e24bdaaf6c875d2625981d'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:30.216191 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-30ce03df-56fe-40bd-bcc4-08af37353ba6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e22e55ca1e24bdaaf6c875d2625981d'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:30.216997 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-30ce03df-56fe-40bd-bcc4-08af37353ba6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e22e55ca1e24bdaaf6c875d2625981d'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:30.229302 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-30ce03df-56fe-40bd-bcc4-08af37353ba6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e22e55ca1e24bdaaf6c875d2625981d'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:30.242598 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 935/1862] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Aug 6 01:33:30 2022] DELETE /identity/v3/users/0e22e55ca1e24bdaaf6c875d2625981d => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:33:30.247217 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53f16ee-7758-4fbd-b9fa-da26f8f20660 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:30.247535 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53f16ee-7758-4fbd-b9fa-da26f8f20660 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:30.247818 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53f16ee-7758-4fbd-b9fa-da26f8f20660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:30.266564 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-f53f16ee-7758-4fbd-b9fa-da26f8f20660 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:33:30.267802 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-f53f16ee-7758-4fbd-b9fa-da26f8f20660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:30.275843 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=86,keystone:INSERT=7,keystone:DELETE=7 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:33:30.328397 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 928/1863] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:33:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:33:30.338779 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ce03df-56fe-40bd-bcc4-08af37353ba6 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:30.340306 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-30ce03df-56fe-40bd-bcc4-08af37353ba6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:30.407102 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ce03df-56fe-40bd-bcc4-08af37353ba6 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:30.411493 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef98973e-143d-4928-8028-df88a133f4cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:30.412123 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef98973e-143d-4928-8028-df88a133f4cf service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:30.412408 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef98973e-143d-4928-8028-df88a133f4cf service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:30.412684 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef98973e-143d-4928-8028-df88a133f4cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:30.422931 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef98973e-143d-4928-8028-df88a133f4cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:30.425108 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef98973e-143d-4928-8028-df88a133f4cf service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:30.434952 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 936/1864] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:33:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:33:30.841932 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53f16ee-7758-4fbd-b9fa-da26f8f20660 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:30.843920 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-f53f16ee-7758-4fbd-b9fa-da26f8f20660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:30.856655 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53f16ee-7758-4fbd-b9fa-da26f8f20660 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:30.861560 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e4fc5c-eede-4823-8edb-f4d643c36f18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:30.862842 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e4fc5c-eede-4823-8edb-f4d643c36f18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:30.863149 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e4fc5c-eede-4823-8edb-f4d643c36f18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:30.863450 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e4fc5c-eede-4823-8edb-f4d643c36f18 admin admin] PATH_INFO: `/v3/projects/8bfb1d46de784327ae9a18cc5305f1da` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:30.864893 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e4fc5c-eede-4823-8edb-f4d643c36f18 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8bfb1d46de784327ae9a18cc5305f1da)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:30.866907 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e4fc5c-eede-4823-8edb-f4d643c36f18 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:30.929708 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-91e4fc5c-eede-4823-8edb-f4d643c36f18 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8bfb1d46de784327ae9a18cc5305f1da', 'request_id': 'req-91e4fc5c-eede-4823-8edb-f4d643c36f18'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:30.935718 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-91e4fc5c-eede-4823-8edb-f4d643c36f18 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8bfb1d46de784327ae9a18cc5305f1da', 'request_id': 'req-91e4fc5c-eede-4823-8edb-f4d643c36f18'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:30.941203 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-91e4fc5c-eede-4823-8edb-f4d643c36f18 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8bfb1d46de784327ae9a18cc5305f1da', 'request_id': 'req-91e4fc5c-eede-4823-8edb-f4d643c36f18'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:30.953097 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-91e4fc5c-eede-4823-8edb-f4d643c36f18 admin admin] The token cache is being invalidate because project 8bfb1d46de784327ae9a18cc5305f1da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:33:30.953389 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-91e4fc5c-eede-4823-8edb-f4d643c36f18 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:30.954043 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-91e4fc5c-eede-4823-8edb-f4d643c36f18 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:30.955826 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 929/1865] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Aug 6 01:33:30 2022] DELETE /identity/v3/projects/8bfb1d46de784327ae9a18cc5305f1da => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:33:30.959844 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef98973e-143d-4928-8028-df88a133f4cf service neutron] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:30.969328 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef98973e-143d-4928-8028-df88a133f4cf service neutron] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:31.029428 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49bf5b0-eaf3-4853-92ba-c292943593ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:31.030080 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49bf5b0-eaf3-4853-92ba-c292943593ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:31.030366 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49bf5b0-eaf3-4853-92ba-c292943593ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:31.030642 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49bf5b0-eaf3-4853-92ba-c292943593ae admin admin] PATH_INFO: `/v3/users/7929a339bb034023816487487ba93886` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:31.031826 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49bf5b0-eaf3-4853-92ba-c292943593ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=7929a339bb034023816487487ba93886)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:31.034586 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49bf5b0-eaf3-4853-92ba-c292943593ae admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:31.081408 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-f49bf5b0-eaf3-4853-92ba-c292943593ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7929a339bb034023816487487ba93886'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:31.088189 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-f49bf5b0-eaf3-4853-92ba-c292943593ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7929a339bb034023816487487ba93886'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:31.088851 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-f49bf5b0-eaf3-4853-92ba-c292943593ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7929a339bb034023816487487ba93886'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:31.089495 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-f49bf5b0-eaf3-4853-92ba-c292943593ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7929a339bb034023816487487ba93886'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:31.100705 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:33:31.101938 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-f49bf5b0-eaf3-4853-92ba-c292943593ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7929a339bb034023816487487ba93886'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:31.116621 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 937/1866] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Aug 6 01:33:30 2022] DELETE /identity/v3/users/7929a339bb034023816487487ba93886 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:33:31.390581 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e4fc5c-eede-4823-8edb-f4d643c36f18 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:31.397184 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-91e4fc5c-eede-4823-8edb-f4d643c36f18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:31.411907 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e4fc5c-eede-4823-8edb-f4d643c36f18 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:31.469258 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13bcc36-6d5a-43a9-8ef7-8698b4df2d75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:31.469911 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13bcc36-6d5a-43a9-8ef7-8698b4df2d75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:31.470194 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13bcc36-6d5a-43a9-8ef7-8698b4df2d75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:31.470473 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13bcc36-6d5a-43a9-8ef7-8698b4df2d75 admin admin] PATH_INFO: `/v3/projects/e1bb0f8e5e2f43ee92b7c00fa2f029ab` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:31.472453 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13bcc36-6d5a-43a9-8ef7-8698b4df2d75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1bb0f8e5e2f43ee92b7c00fa2f029ab)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:31.474404 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13bcc36-6d5a-43a9-8ef7-8698b4df2d75 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:31.516516 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:33:31.527025 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-b13bcc36-6d5a-43a9-8ef7-8698b4df2d75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1bb0f8e5e2f43ee92b7c00fa2f029ab', 'request_id': 'req-b13bcc36-6d5a-43a9-8ef7-8698b4df2d75'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:31.533384 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-b13bcc36-6d5a-43a9-8ef7-8698b4df2d75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1bb0f8e5e2f43ee92b7c00fa2f029ab', 'request_id': 'req-b13bcc36-6d5a-43a9-8ef7-8698b4df2d75'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:31.539495 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-b13bcc36-6d5a-43a9-8ef7-8698b4df2d75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1bb0f8e5e2f43ee92b7c00fa2f029ab', 'request_id': 'req-b13bcc36-6d5a-43a9-8ef7-8698b4df2d75'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:31.554296 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-b13bcc36-6d5a-43a9-8ef7-8698b4df2d75 admin admin] The token cache is being invalidate because project e1bb0f8e5e2f43ee92b7c00fa2f029ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:33:31.554881 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-b13bcc36-6d5a-43a9-8ef7-8698b4df2d75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:31.555632 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-b13bcc36-6d5a-43a9-8ef7-8698b4df2d75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:31.557559 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 930/1867] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Aug 6 01:33:31 2022] DELETE /identity/v3/projects/e1bb0f8e5e2f43ee92b7c00fa2f029ab => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:33:31.562183 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbdadb0-f130-4723-9a56-0ebeff451fb6 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:31.562489 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbdadb0-f130-4723-9a56-0ebeff451fb6 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:31.563417 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbdadb0-f130-4723-9a56-0ebeff451fb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:31.583075 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-0fbdadb0-f130-4723-9a56-0ebeff451fb6 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:33:31.585098 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-0fbdadb0-f130-4723-9a56-0ebeff451fb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:31.655698 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 938/1868] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:33:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:33:31.663944 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc9d043-bc9b-4cf8-8830-19abd541e47f None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:31.664239 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc9d043-bc9b-4cf8-8830-19abd541e47f None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:31.664520 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc9d043-bc9b-4cf8-8830-19abd541e47f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:31.684500 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-9dc9d043-bc9b-4cf8-8830-19abd541e47f None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:33:31.686465 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-9dc9d043-bc9b-4cf8-8830-19abd541e47f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:31.694583 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 931/1869] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:33:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:33:31.698830 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbdadb0-f130-4723-9a56-0ebeff451fb6 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:31.709895 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbdadb0-f130-4723-9a56-0ebeff451fb6 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:31.714913 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778be01f-fd38-4fbc-871f-4a02124637de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:31.715877 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778be01f-fd38-4fbc-871f-4a02124637de admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:31.716284 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778be01f-fd38-4fbc-871f-4a02124637de admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:31.716621 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778be01f-fd38-4fbc-871f-4a02124637de admin admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:31.717457 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778be01f-fd38-4fbc-871f-4a02124637de admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:33:31.717861 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778be01f-fd38-4fbc-871f-4a02124637de admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:31.719766 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778be01f-fd38-4fbc-871f-4a02124637de admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:31.734248 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 939/1870] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Aug 6 01:33:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:33:31.740635 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc9d043-bc9b-4cf8-8830-19abd541e47f None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:31.750121 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc9d043-bc9b-4cf8-8830-19abd541e47f None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:31.755336 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3e53b6-4671-4d0f-b36e-9918d44a9ac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:31.756050 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3e53b6-4671-4d0f-b36e-9918d44a9ac4 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:31.756408 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3e53b6-4671-4d0f-b36e-9918d44a9ac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:31.756755 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3e53b6-4671-4d0f-b36e-9918d44a9ac4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:31.757629 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3e53b6-4671-4d0f-b36e-9918d44a9ac4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:31.759647 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3e53b6-4671-4d0f-b36e-9918d44a9ac4 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:31.792783 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 932/1871] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Aug 6 01:33:31 2022] POST /identity/v3/projects => generated 387 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:33:31.794417 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778be01f-fd38-4fbc-871f-4a02124637de admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:31.807802 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778be01f-fd38-4fbc-871f-4a02124637de admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:31.882783 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afce807-49da-4bcb-8fde-eb796adc8fec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:31.883411 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afce807-49da-4bcb-8fde-eb796adc8fec admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:31.883696 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afce807-49da-4bcb-8fde-eb796adc8fec admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:31.883969 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afce807-49da-4bcb-8fde-eb796adc8fec admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:31.884644 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afce807-49da-4bcb-8fde-eb796adc8fec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:31.886609 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afce807-49da-4bcb-8fde-eb796adc8fec admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:31.906273 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 940/1872] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:33:31 2022] POST /identity/v3/users => generated 294 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:33:31.911034 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3e53b6-4671-4d0f-b36e-9918d44a9ac4 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:31.922927 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3e53b6-4671-4d0f-b36e-9918d44a9ac4 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:31.929471 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96622f73-326b-4eb8-92c3-781e6ff8cfb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:31.930165 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96622f73-326b-4eb8-92c3-781e6ff8cfb3 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:31.930454 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96622f73-326b-4eb8-92c3-781e6ff8cfb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:31.930731 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96622f73-326b-4eb8-92c3-781e6ff8cfb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:31.931483 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96622f73-326b-4eb8-92c3-781e6ff8cfb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:31.933920 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96622f73-326b-4eb8-92c3-781e6ff8cfb3 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:31.947033 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 933/1873] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:33:31 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 01:33:31.951543 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afce807-49da-4bcb-8fde-eb796adc8fec admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:31.962706 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afce807-49da-4bcb-8fde-eb796adc8fec admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:31.967484 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccba636b-7503-41bb-a436-01e2ab245b91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:31.968102 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccba636b-7503-41bb-a436-01e2ab245b91 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:31.968383 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccba636b-7503-41bb-a436-01e2ab245b91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:31.968658 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccba636b-7503-41bb-a436-01e2ab245b91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:31.969404 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccba636b-7503-41bb-a436-01e2ab245b91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:31.971698 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccba636b-7503-41bb-a436-01e2ab245b91 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:31.982666 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 941/1874] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:33:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:33:31.987428 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96622f73-326b-4eb8-92c3-781e6ff8cfb3 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:31.999345 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96622f73-326b-4eb8-92c3-781e6ff8cfb3 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:32.004227 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c32a6d-470e-40e7-a61d-3d75fd0ca0f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:32.004909 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c32a6d-470e-40e7-a61d-3d75fd0ca0f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:32.005243 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c32a6d-470e-40e7-a61d-3d75fd0ca0f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:32.005524 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c32a6d-470e-40e7-a61d-3d75fd0ca0f2 admin admin] PATH_INFO: `/v3/projects/f54e92685246476f9c1268dc4b92cdce/users/8b44fc5ab0744bc78cfbe5c96dc4c372/roles/61b3b93d37a047e4bb8369b46fe20db9` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:32.029708 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c32a6d-470e-40e7-a61d-3d75fd0ca0f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f54e92685246476f9c1268dc4b92cdce, user_id=8b44fc5ab0744bc78cfbe5c96dc4c372, role_id=61b3b93d37a047e4bb8369b46fe20db9)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:32.034150 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c32a6d-470e-40e7-a61d-3d75fd0ca0f2 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:32.051757 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 934/1875] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:33:31 2022] PUT /identity/v3/projects/f54e92685246476f9c1268dc4b92cdce/users/8b44fc5ab0744bc78cfbe5c96dc4c372/roles/61b3b93d37a047e4bb8369b46fe20db9 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:33:32.062260 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dd49f2-b695-452d-aa57-ef5cd4bb8ab6 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:32.064178 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dd49f2-b695-452d-aa57-ef5cd4bb8ab6 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:32.064475 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dd49f2-b695-452d-aa57-ef5cd4bb8ab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:32.106398 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-75dd49f2-b695-452d-aa57-ef5cd4bb8ab6 None None] MFA Rules not processed for user `8b44fc5ab0744bc78cfbe5c96dc4c372`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:33:32.108424 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-75dd49f2-b695-452d-aa57-ef5cd4bb8ab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:32.164999 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:33:32.208419 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 942/1876] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:33:32 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:33:32.212504 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c32a6d-470e-40e7-a61d-3d75fd0ca0f2 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:32.275446 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c32a6d-470e-40e7-a61d-3d75fd0ca0f2 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:32.279973 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31fff06-666f-4585-b33f-b12533dcc74f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:32.280610 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31fff06-666f-4585-b33f-b12533dcc74f admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:32.280887 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31fff06-666f-4585-b33f-b12533dcc74f admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:32.281190 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31fff06-666f-4585-b33f-b12533dcc74f admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:32.281853 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31fff06-666f-4585-b33f-b12533dcc74f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:32.284181 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31fff06-666f-4585-b33f-b12533dcc74f admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:32.310420 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 935/1877] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Aug 6 01:33:32 2022] POST /identity/v3/projects => generated 385 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:33:32.315422 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dd49f2-b695-452d-aa57-ef5cd4bb8ab6 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:32.325983 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dd49f2-b695-452d-aa57-ef5cd4bb8ab6 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:32.382054 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442e2969-5bb2-4e38-bfcc-771c437abeb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:32.382669 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442e2969-5bb2-4e38-bfcc-771c437abeb2 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:32.382951 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442e2969-5bb2-4e38-bfcc-771c437abeb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:32.383225 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442e2969-5bb2-4e38-bfcc-771c437abeb2 admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:32.383893 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442e2969-5bb2-4e38-bfcc-771c437abeb2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:32.385830 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442e2969-5bb2-4e38-bfcc-771c437abeb2 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:32.410835 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 943/1878] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:33:32 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:33:32.411960 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31fff06-666f-4585-b33f-b12533dcc74f admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:32.425401 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31fff06-666f-4585-b33f-b12533dcc74f admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:32.430073 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2636af44-58ee-4733-ba4c-98b1b6a04a04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:32.430689 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2636af44-58ee-4733-ba4c-98b1b6a04a04 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:32.430967 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2636af44-58ee-4733-ba4c-98b1b6a04a04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:32.431240 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2636af44-58ee-4733-ba4c-98b1b6a04a04 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:32.431929 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2636af44-58ee-4733-ba4c-98b1b6a04a04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:32.433541 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2636af44-58ee-4733-ba4c-98b1b6a04a04 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:32.444488 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 936/1879] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:33:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 01:33:32.448410 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442e2969-5bb2-4e38-bfcc-771c437abeb2 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:32.458324 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442e2969-5bb2-4e38-bfcc-771c437abeb2 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:32.462883 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610b6b17-c372-40e4-8703-d4f69844b304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:32.463484 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610b6b17-c372-40e4-8703-d4f69844b304 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:32.463762 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610b6b17-c372-40e4-8703-d4f69844b304 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:32.464037 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610b6b17-c372-40e4-8703-d4f69844b304 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:32.464724 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610b6b17-c372-40e4-8703-d4f69844b304 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:32.466348 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610b6b17-c372-40e4-8703-d4f69844b304 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:32.477911 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 944/1880] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:33:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:33:32.482813 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2636af44-58ee-4733-ba4c-98b1b6a04a04 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:32.496152 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2636af44-58ee-4733-ba4c-98b1b6a04a04 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:32.500696 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1448102d-73dd-4d8e-ae91-c764d8239ab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:32.501402 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1448102d-73dd-4d8e-ae91-c764d8239ab9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:32.501682 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1448102d-73dd-4d8e-ae91-c764d8239ab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:32.501958 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1448102d-73dd-4d8e-ae91-c764d8239ab9 admin admin] PATH_INFO: `/v3/projects/966e80d40b844b659e8c1fab09016f6f/users/a3a2f9b5208442ce803e98fcc9334693/roles/61b3b93d37a047e4bb8369b46fe20db9` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:32.517079 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1448102d-73dd-4d8e-ae91-c764d8239ab9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=966e80d40b844b659e8c1fab09016f6f, user_id=a3a2f9b5208442ce803e98fcc9334693, role_id=61b3b93d37a047e4bb8369b46fe20db9)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:32.520091 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1448102d-73dd-4d8e-ae91-c764d8239ab9 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:32.532351 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 937/1881] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:33:32 2022] PUT /identity/v3/projects/966e80d40b844b659e8c1fab09016f6f/users/a3a2f9b5208442ce803e98fcc9334693/roles/61b3b93d37a047e4bb8369b46fe20db9 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:33:32.544371 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0110501-1401-4c35-825c-4e7134d9bccc None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:32.544664 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0110501-1401-4c35-825c-4e7134d9bccc None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:32.544944 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0110501-1401-4c35-825c-4e7134d9bccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:32.573352 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-e0110501-1401-4c35-825c-4e7134d9bccc None None] MFA Rules not processed for user `a3a2f9b5208442ce803e98fcc9334693`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:33:32.574553 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-e0110501-1401-4c35-825c-4e7134d9bccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:32.639714 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 945/1882] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:33:32 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:33:32.644531 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1448102d-73dd-4d8e-ae91-c764d8239ab9 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:32.658059 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:33:32.712922 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1448102d-73dd-4d8e-ae91-c764d8239ab9 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:32.717578 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc5ecf8-db4c-4239-a2e1-d71943d32b42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:32.718190 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc5ecf8-db4c-4239-a2e1-d71943d32b42 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:32.718468 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc5ecf8-db4c-4239-a2e1-d71943d32b42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:32.718739 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc5ecf8-db4c-4239-a2e1-d71943d32b42 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:32.719406 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc5ecf8-db4c-4239-a2e1-d71943d32b42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:32.722160 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc5ecf8-db4c-4239-a2e1-d71943d32b42 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:32.749095 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 938/1883] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Aug 6 01:33:32 2022] POST /identity/v3/projects => generated 385 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:33:32.753174 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0110501-1401-4c35-825c-4e7134d9bccc None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:32.764188 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0110501-1401-4c35-825c-4e7134d9bccc None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:32.813816 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:33:32.824136 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3d2fa4-6b58-4634-9b84-1ded9b34023a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:32.824876 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3d2fa4-6b58-4634-9b84-1ded9b34023a admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:32.825198 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3d2fa4-6b58-4634-9b84-1ded9b34023a admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:32.825479 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3d2fa4-6b58-4634-9b84-1ded9b34023a admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:32.826151 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3d2fa4-6b58-4634-9b84-1ded9b34023a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:32.828944 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3d2fa4-6b58-4634-9b84-1ded9b34023a admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:32.872968 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 946/1884] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:33:32 2022] POST /identity/v3/users => generated 299 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:33:32.876940 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc5ecf8-db4c-4239-a2e1-d71943d32b42 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:32.888638 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc5ecf8-db4c-4239-a2e1-d71943d32b42 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:32.896227 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18879861-20e5-4d7c-a24c-244970828819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:32.896874 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18879861-20e5-4d7c-a24c-244970828819 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:32.897192 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18879861-20e5-4d7c-a24c-244970828819 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:32.897477 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18879861-20e5-4d7c-a24c-244970828819 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:32.898542 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18879861-20e5-4d7c-a24c-244970828819 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:32.900059 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18879861-20e5-4d7c-a24c-244970828819 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:32.912912 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 939/1885] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:33:32 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 01:33:32.917366 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3d2fa4-6b58-4634-9b84-1ded9b34023a admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:32.929670 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3d2fa4-6b58-4634-9b84-1ded9b34023a admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:32.936999 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31935ec6-ecc5-4e5e-acc0-4aeaf93e7b54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:32.937790 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31935ec6-ecc5-4e5e-acc0-4aeaf93e7b54 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:32.938262 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31935ec6-ecc5-4e5e-acc0-4aeaf93e7b54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:32.938659 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31935ec6-ecc5-4e5e-acc0-4aeaf93e7b54 admin admin] PATH_INFO: `/v3/projects/379251fceb7740db9b5520d4cbd0dee1/users/bd5eb2d5d9b14354a8c4564ca85732d2/roles/4559b9a0837249f895a13c70096f8d38` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:32.964185 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31935ec6-ecc5-4e5e-acc0-4aeaf93e7b54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=379251fceb7740db9b5520d4cbd0dee1, user_id=bd5eb2d5d9b14354a8c4564ca85732d2, role_id=4559b9a0837249f895a13c70096f8d38)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:32.968231 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31935ec6-ecc5-4e5e-acc0-4aeaf93e7b54 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:32.993215 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 947/1886] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:33:32 2022] PUT /identity/v3/projects/379251fceb7740db9b5520d4cbd0dee1/users/bd5eb2d5d9b14354a8c4564ca85732d2/roles/4559b9a0837249f895a13c70096f8d38 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:33:32.994162 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18879861-20e5-4d7c-a24c-244970828819 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:33.005394 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18879861-20e5-4d7c-a24c-244970828819 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:33.083054 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f71ffa-4709-4806-87f5-90c771401b27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:33.083815 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f71ffa-4709-4806-87f5-90c771401b27 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:33.084410 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f71ffa-4709-4806-87f5-90c771401b27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:33.084704 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f71ffa-4709-4806-87f5-90c771401b27 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:33.085439 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f71ffa-4709-4806-87f5-90c771401b27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:33.087067 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f71ffa-4709-4806-87f5-90c771401b27 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:33.101137 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 940/1887] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:33:32 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 01:33:33.102890 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31935ec6-ecc5-4e5e-acc0-4aeaf93e7b54 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:33.115363 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31935ec6-ecc5-4e5e-acc0-4aeaf93e7b54 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:33.120505 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4ebc77-7021-406c-867a-cad58c9384a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:33.121235 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4ebc77-7021-406c-867a-cad58c9384a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:33.121518 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4ebc77-7021-406c-867a-cad58c9384a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:33.121804 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4ebc77-7021-406c-867a-cad58c9384a0 admin admin] PATH_INFO: `/v3/projects/379251fceb7740db9b5520d4cbd0dee1/users/bd5eb2d5d9b14354a8c4564ca85732d2/roles/4559b9a0837249f895a13c70096f8d38` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:33.123811 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4ebc77-7021-406c-867a-cad58c9384a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=379251fceb7740db9b5520d4cbd0dee1, user_id=bd5eb2d5d9b14354a8c4564ca85732d2, role_id=4559b9a0837249f895a13c70096f8d38)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:33.126846 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4ebc77-7021-406c-867a-cad58c9384a0 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:33.140118 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 948/1888] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:33:33 2022] PUT /identity/v3/projects/379251fceb7740db9b5520d4cbd0dee1/users/bd5eb2d5d9b14354a8c4564ca85732d2/roles/4559b9a0837249f895a13c70096f8d38 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:33:33.151385 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04de41ed-5abd-4cb9-bddf-d3f2c2182c2d None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:33.151684 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04de41ed-5abd-4cb9-bddf-d3f2c2182c2d None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:33.152065 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04de41ed-5abd-4cb9-bddf-d3f2c2182c2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:33.192790 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-04de41ed-5abd-4cb9-bddf-d3f2c2182c2d None None] MFA Rules not processed for user `bd5eb2d5d9b14354a8c4564ca85732d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:33:33.194062 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-04de41ed-5abd-4cb9-bddf-d3f2c2182c2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:33.213803 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203cf65c-edc0-4aec-a4af-0359ac3138da None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:33.214100 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203cf65c-edc0-4aec-a4af-0359ac3138da None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:33.214441 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203cf65c-edc0-4aec-a4af-0359ac3138da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:33.247114 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-203cf65c-edc0-4aec-a4af-0359ac3138da None None] MFA Rules not processed for user `3f081e5336f74e21903f11df0b9d95b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:33:33.248465 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-203cf65c-edc0-4aec-a4af-0359ac3138da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:33.306387 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 941/1889] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:33:33 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:33:33.321182 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04de41ed-5abd-4cb9-bddf-d3f2c2182c2d None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:33.322656 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-04de41ed-5abd-4cb9-bddf-d3f2c2182c2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:33.323057 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 949/1890] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:33:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:33:33.334975 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:33:33.386006 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04de41ed-5abd-4cb9-bddf-d3f2c2182c2d None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:33.389773 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe41efe-9d63-42e3-930e-cf02336eb583 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1be1e6173e374a60b9e1fbf51a1ab571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:33.390411 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe41efe-9d63-42e3-930e-cf02336eb583 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:33.390693 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe41efe-9d63-42e3-930e-cf02336eb583 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:33.390970 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe41efe-9d63-42e3-930e-cf02336eb583 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:33.401725 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe41efe-9d63-42e3-930e-cf02336eb583 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:33.403189 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe41efe-9d63-42e3-930e-cf02336eb583 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:33.414081 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 942/1891] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:33:33 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 01:33:33.966490 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203cf65c-edc0-4aec-a4af-0359ac3138da None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:33.967986 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-203cf65c-edc0-4aec-a4af-0359ac3138da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:33.981104 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203cf65c-edc0-4aec-a4af-0359ac3138da None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:33.984744 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edecbb2-31da-43c8-9d16-f3d582a10344 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1be1e6173e374a60b9e1fbf51a1ab571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:33.985394 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edecbb2-31da-43c8-9d16-f3d582a10344 service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:33.985679 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edecbb2-31da-43c8-9d16-f3d582a10344 service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:33.985955 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edecbb2-31da-43c8-9d16-f3d582a10344 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:34.061645 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edecbb2-31da-43c8-9d16-f3d582a10344 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:34.063073 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edecbb2-31da-43c8-9d16-f3d582a10344 service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:34.077598 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 950/1892] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:33:33 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:33:39.921037 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe41efe-9d63-42e3-930e-cf02336eb583 service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:39.934002 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe41efe-9d63-42e3-930e-cf02336eb583 service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:39.937690 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9263b34a-0534-4247-96f4-cc55f99bec34 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1be1e6173e374a60b9e1fbf51a1ab571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:39.938315 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9263b34a-0534-4247-96f4-cc55f99bec34 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:39.938600 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9263b34a-0534-4247-96f4-cc55f99bec34 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:39.938876 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9263b34a-0534-4247-96f4-cc55f99bec34 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:39.941023 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-9263b34a-0534-4247-96f4-cc55f99bec34 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:40.016099 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9263b34a-0534-4247-96f4-cc55f99bec34 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:40.018339 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9263b34a-0534-4247-96f4-cc55f99bec34 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:40.046455 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 943/1893] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:33:39 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 01:33:41.475620 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edecbb2-31da-43c8-9d16-f3d582a10344 service cinder] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:41.484928 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edecbb2-31da-43c8-9d16-f3d582a10344 service cinder] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:41.490244 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece69454-108e-431d-b1c4-1921085093bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1be1e6173e374a60b9e1fbf51a1ab571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:41.490886 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece69454-108e-431d-b1c4-1921085093bb service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:41.491174 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece69454-108e-431d-b1c4-1921085093bb service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:41.491453 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece69454-108e-431d-b1c4-1921085093bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:41.541681 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:33:41.556227 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece69454-108e-431d-b1c4-1921085093bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:41.557947 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece69454-108e-431d-b1c4-1921085093bb service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:41.569429 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 951/1894] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:33:41 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:33:43.979495 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9263b34a-0534-4247-96f4-cc55f99bec34 service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:44.043972 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9263b34a-0534-4247-96f4-cc55f99bec34 service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:44.048709 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f31d5df-5531-48e2-b90e-535d3f162c6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:44.049570 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f31d5df-5531-48e2-b90e-535d3f162c6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:44.049855 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f31d5df-5531-48e2-b90e-535d3f162c6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:44.050138 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f31d5df-5531-48e2-b90e-535d3f162c6d admin admin] PATH_INFO: `/v3/users/8b44fc5ab0744bc78cfbe5c96dc4c372` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:44.051325 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f31d5df-5531-48e2-b90e-535d3f162c6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b44fc5ab0744bc78cfbe5c96dc4c372)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:44.053291 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f31d5df-5531-48e2-b90e-535d3f162c6d admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:44.100954 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-8f31d5df-5531-48e2-b90e-535d3f162c6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b44fc5ab0744bc78cfbe5c96dc4c372'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:44.101759 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-8f31d5df-5531-48e2-b90e-535d3f162c6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b44fc5ab0744bc78cfbe5c96dc4c372'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:44.102378 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-8f31d5df-5531-48e2-b90e-535d3f162c6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b44fc5ab0744bc78cfbe5c96dc4c372'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:44.103574 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:33:44.116758 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-8f31d5df-5531-48e2-b90e-535d3f162c6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b44fc5ab0744bc78cfbe5c96dc4c372'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:44.123519 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-8f31d5df-5531-48e2-b90e-535d3f162c6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b44fc5ab0744bc78cfbe5c96dc4c372'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:44.137842 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 944/1895] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Aug 6 01:33:43 2022] DELETE /identity/v3/users/8b44fc5ab0744bc78cfbe5c96dc4c372 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:33:44.143378 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7320040-9f0d-4bc0-acb3-0d3f99c2153d None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:44.143840 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7320040-9f0d-4bc0-acb3-0d3f99c2153d None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:44.144179 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7320040-9f0d-4bc0-acb3-0d3f99c2153d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:44.163870 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-a7320040-9f0d-4bc0-acb3-0d3f99c2153d None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:33:44.165238 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-a7320040-9f0d-4bc0-acb3-0d3f99c2153d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:44.240904 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 952/1896] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:33:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:33:44.253411 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f31d5df-5531-48e2-b90e-535d3f162c6d admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:44.255516 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-8f31d5df-5531-48e2-b90e-535d3f162c6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:44.329108 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f31d5df-5531-48e2-b90e-535d3f162c6d admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:44.333151 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf530b2d-f575-4bf2-a950-532ff7b1d067 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:44.333776 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf530b2d-f575-4bf2-a950-532ff7b1d067 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:44.334096 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf530b2d-f575-4bf2-a950-532ff7b1d067 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:44.334346 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf530b2d-f575-4bf2-a950-532ff7b1d067 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:44.344029 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf530b2d-f575-4bf2-a950-532ff7b1d067 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:44.346081 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf530b2d-f575-4bf2-a950-532ff7b1d067 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:44.359976 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 945/1897] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:33:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 01:33:44.674732 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7320040-9f0d-4bc0-acb3-0d3f99c2153d None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:44.676411 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-a7320040-9f0d-4bc0-acb3-0d3f99c2153d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:44.690520 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7320040-9f0d-4bc0-acb3-0d3f99c2153d None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:44.698922 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2127783d-0b35-4ecb-a7f5-1ecac0cdc3fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:44.700245 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2127783d-0b35-4ecb-a7f5-1ecac0cdc3fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:44.700533 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2127783d-0b35-4ecb-a7f5-1ecac0cdc3fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:44.700808 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2127783d-0b35-4ecb-a7f5-1ecac0cdc3fb admin admin] PATH_INFO: `/v3/projects/f54e92685246476f9c1268dc4b92cdce` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:44.703057 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2127783d-0b35-4ecb-a7f5-1ecac0cdc3fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=f54e92685246476f9c1268dc4b92cdce)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:44.705756 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2127783d-0b35-4ecb-a7f5-1ecac0cdc3fb admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:44.766714 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-2127783d-0b35-4ecb-a7f5-1ecac0cdc3fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f54e92685246476f9c1268dc4b92cdce', 'request_id': 'req-2127783d-0b35-4ecb-a7f5-1ecac0cdc3fb'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:44.773428 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-2127783d-0b35-4ecb-a7f5-1ecac0cdc3fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f54e92685246476f9c1268dc4b92cdce', 'request_id': 'req-2127783d-0b35-4ecb-a7f5-1ecac0cdc3fb'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:44.779256 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-2127783d-0b35-4ecb-a7f5-1ecac0cdc3fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f54e92685246476f9c1268dc4b92cdce', 'request_id': 'req-2127783d-0b35-4ecb-a7f5-1ecac0cdc3fb'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:44.793115 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-2127783d-0b35-4ecb-a7f5-1ecac0cdc3fb admin admin] The token cache is being invalidate because project f54e92685246476f9c1268dc4b92cdce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:33:44.793411 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-2127783d-0b35-4ecb-a7f5-1ecac0cdc3fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:44.794044 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-2127783d-0b35-4ecb-a7f5-1ecac0cdc3fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:44.795724 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 953/1898] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Aug 6 01:33:44 2022] DELETE /identity/v3/projects/f54e92685246476f9c1268dc4b92cdce => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:33:44.799725 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf530b2d-f575-4bf2-a950-532ff7b1d067 service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:44.817472 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf530b2d-f575-4bf2-a950-532ff7b1d067 service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:44.886136 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd06112e-c78b-453d-9e8e-4368eac75969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:44.886852 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd06112e-c78b-453d-9e8e-4368eac75969 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:44.887192 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd06112e-c78b-453d-9e8e-4368eac75969 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:44.887410 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd06112e-c78b-453d-9e8e-4368eac75969 admin admin] PATH_INFO: `/v3/users/a3a2f9b5208442ce803e98fcc9334693` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:44.888736 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd06112e-c78b-453d-9e8e-4368eac75969 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3a2f9b5208442ce803e98fcc9334693)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:44.890939 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd06112e-c78b-453d-9e8e-4368eac75969 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:44.954049 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-cd06112e-c78b-453d-9e8e-4368eac75969 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3a2f9b5208442ce803e98fcc9334693'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:44.954713 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-cd06112e-c78b-453d-9e8e-4368eac75969 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3a2f9b5208442ce803e98fcc9334693'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:44.955558 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-cd06112e-c78b-453d-9e8e-4368eac75969 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3a2f9b5208442ce803e98fcc9334693'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:44.976212 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-cd06112e-c78b-453d-9e8e-4368eac75969 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3a2f9b5208442ce803e98fcc9334693'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:44.982833 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:33:44.984977 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-cd06112e-c78b-453d-9e8e-4368eac75969 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3a2f9b5208442ce803e98fcc9334693'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:45.015992 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 946/1899] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Aug 6 01:33:44 2022] DELETE /identity/v3/users/a3a2f9b5208442ce803e98fcc9334693 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:33:45.666402 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2127783d-0b35-4ecb-a7f5-1ecac0cdc3fb admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:45.667886 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-2127783d-0b35-4ecb-a7f5-1ecac0cdc3fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:45.683382 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2127783d-0b35-4ecb-a7f5-1ecac0cdc3fb admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:45.750152 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a4361d-64c8-4bde-92a3-80ae04ecc06e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:45.750809 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a4361d-64c8-4bde-92a3-80ae04ecc06e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:45.751096 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a4361d-64c8-4bde-92a3-80ae04ecc06e admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:45.751377 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a4361d-64c8-4bde-92a3-80ae04ecc06e admin admin] PATH_INFO: `/v3/projects/966e80d40b844b659e8c1fab09016f6f` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:45.752651 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a4361d-64c8-4bde-92a3-80ae04ecc06e admin admin] RBAC: Authorizing `identity:delete_project(project_id=966e80d40b844b659e8c1fab09016f6f)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:45.754602 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a4361d-64c8-4bde-92a3-80ae04ecc06e admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:45.767310 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:33:45.816510 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-c5a4361d-64c8-4bde-92a3-80ae04ecc06e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '966e80d40b844b659e8c1fab09016f6f', 'request_id': 'req-c5a4361d-64c8-4bde-92a3-80ae04ecc06e'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:45.832795 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-c5a4361d-64c8-4bde-92a3-80ae04ecc06e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '966e80d40b844b659e8c1fab09016f6f', 'request_id': 'req-c5a4361d-64c8-4bde-92a3-80ae04ecc06e'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:45.840561 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-c5a4361d-64c8-4bde-92a3-80ae04ecc06e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '966e80d40b844b659e8c1fab09016f6f', 'request_id': 'req-c5a4361d-64c8-4bde-92a3-80ae04ecc06e'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:45.859037 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-c5a4361d-64c8-4bde-92a3-80ae04ecc06e admin admin] The token cache is being invalidate because project 966e80d40b844b659e8c1fab09016f6f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:33:45.859716 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-c5a4361d-64c8-4bde-92a3-80ae04ecc06e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:45.865574 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-c5a4361d-64c8-4bde-92a3-80ae04ecc06e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:45.873668 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 954/1900] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Aug 6 01:33:45 2022] DELETE /identity/v3/projects/966e80d40b844b659e8c1fab09016f6f => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:33:45.878895 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd06112e-c78b-453d-9e8e-4368eac75969 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:45.880564 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-cd06112e-c78b-453d-9e8e-4368eac75969 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:45.894527 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd06112e-c78b-453d-9e8e-4368eac75969 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:45.959223 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b0b3af-acc2-4dac-992d-27b7fefb273d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:45.959907 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b0b3af-acc2-4dac-992d-27b7fefb273d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:45.960184 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b0b3af-acc2-4dac-992d-27b7fefb273d admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:45.960463 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b0b3af-acc2-4dac-992d-27b7fefb273d admin admin] PATH_INFO: `/v3/users/bd5eb2d5d9b14354a8c4564ca85732d2` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:45.961814 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b0b3af-acc2-4dac-992d-27b7fefb273d admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd5eb2d5d9b14354a8c4564ca85732d2)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:45.963730 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b0b3af-acc2-4dac-992d-27b7fefb273d admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:46.028300 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-04b0b3af-acc2-4dac-992d-27b7fefb273d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd5eb2d5d9b14354a8c4564ca85732d2'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:46.030129 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-04b0b3af-acc2-4dac-992d-27b7fefb273d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd5eb2d5d9b14354a8c4564ca85732d2'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:46.031070 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-04b0b3af-acc2-4dac-992d-27b7fefb273d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd5eb2d5d9b14354a8c4564ca85732d2'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:46.047567 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-04b0b3af-acc2-4dac-992d-27b7fefb273d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd5eb2d5d9b14354a8c4564ca85732d2'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:46.055459 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-04b0b3af-acc2-4dac-992d-27b7fefb273d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd5eb2d5d9b14354a8c4564ca85732d2'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:46.068676 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 947/1901] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Aug 6 01:33:45 2022] DELETE /identity/v3/users/bd5eb2d5d9b14354a8c4564ca85732d2 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:33:46.292647 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a4361d-64c8-4bde-92a3-80ae04ecc06e admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:46.294655 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-c5a4361d-64c8-4bde-92a3-80ae04ecc06e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:46.318007 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a4361d-64c8-4bde-92a3-80ae04ecc06e admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:46.409609 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ab7e17-3e88-4822-ba45-6b6b3d2d0803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:46.410315 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ab7e17-3e88-4822-ba45-6b6b3d2d0803 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:46.413544 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ab7e17-3e88-4822-ba45-6b6b3d2d0803 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:46.414085 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ab7e17-3e88-4822-ba45-6b6b3d2d0803 admin admin] PATH_INFO: `/v3/users/821ef914d521425fb2e558c0d8e8e885` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:46.414281 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b0b3af-acc2-4dac-992d-27b7fefb273d admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:46.415988 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-04b0b3af-acc2-4dac-992d-27b7fefb273d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:46.416223 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ab7e17-3e88-4822-ba45-6b6b3d2d0803 admin admin] RBAC: Authorizing `identity:delete_user(user_id=821ef914d521425fb2e558c0d8e8e885)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:46.418466 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ab7e17-3e88-4822-ba45-6b6b3d2d0803 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:46.442878 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b0b3af-acc2-4dac-992d-27b7fefb273d admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:46.464098 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f88421f-c3e7-47ae-b7d3-96b6a874baa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:46.476730 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f88421f-c3e7-47ae-b7d3-96b6a874baa6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:46.477941 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f88421f-c3e7-47ae-b7d3-96b6a874baa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:46.478320 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f88421f-c3e7-47ae-b7d3-96b6a874baa6 admin admin] PATH_INFO: `/v3/projects/379251fceb7740db9b5520d4cbd0dee1` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:46.479829 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f88421f-c3e7-47ae-b7d3-96b6a874baa6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=379251fceb7740db9b5520d4cbd0dee1)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:46.482227 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f88421f-c3e7-47ae-b7d3-96b6a874baa6 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:46.497941 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-57ab7e17-3e88-4822-ba45-6b6b3d2d0803 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '821ef914d521425fb2e558c0d8e8e885'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:46.509145 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-57ab7e17-3e88-4822-ba45-6b6b3d2d0803 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '821ef914d521425fb2e558c0d8e8e885'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:46.510006 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-57ab7e17-3e88-4822-ba45-6b6b3d2d0803 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '821ef914d521425fb2e558c0d8e8e885'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:46.510731 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-57ab7e17-3e88-4822-ba45-6b6b3d2d0803 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '821ef914d521425fb2e558c0d8e8e885'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:46.533777 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-57ab7e17-3e88-4822-ba45-6b6b3d2d0803 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '821ef914d521425fb2e558c0d8e8e885'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:46.550070 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-5f88421f-c3e7-47ae-b7d3-96b6a874baa6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '379251fceb7740db9b5520d4cbd0dee1', 'request_id': 'req-5f88421f-c3e7-47ae-b7d3-96b6a874baa6'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:46.558309 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-5f88421f-c3e7-47ae-b7d3-96b6a874baa6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '379251fceb7740db9b5520d4cbd0dee1', 'request_id': 'req-5f88421f-c3e7-47ae-b7d3-96b6a874baa6'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:46.574193 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=81,keystone:INSERT=2,keystone:DELETE=17 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:33:46.576753 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 955/1902] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Aug 6 01:33:46 2022] DELETE /identity/v3/users/821ef914d521425fb2e558c0d8e8e885 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:33:46.580140 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-5f88421f-c3e7-47ae-b7d3-96b6a874baa6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '379251fceb7740db9b5520d4cbd0dee1', 'request_id': 'req-5f88421f-c3e7-47ae-b7d3-96b6a874baa6'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:46.581561 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdea956-1fef-4c3c-b95a-723824532db0 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:46.581915 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdea956-1fef-4c3c-b95a-723824532db0 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:46.582256 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdea956-1fef-4c3c-b95a-723824532db0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:46.630605 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-5f88421f-c3e7-47ae-b7d3-96b6a874baa6 admin admin] The token cache is being invalidate because project 379251fceb7740db9b5520d4cbd0dee1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:33:46.630920 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-5f88421f-c3e7-47ae-b7d3-96b6a874baa6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:46.632732 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-5f88421f-c3e7-47ae-b7d3-96b6a874baa6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:46.635043 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-3bdea956-1fef-4c3c-b95a-723824532db0 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:33:46.637980 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 948/1903] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Aug 6 01:33:46 2022] DELETE /identity/v3/projects/379251fceb7740db9b5520d4cbd0dee1 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:33:46.638428 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-3bdea956-1fef-4c3c-b95a-723824532db0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:46.644405 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df07d760-220a-4051-92ce-cf37a0d89d27 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:46.644895 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df07d760-220a-4051-92ce-cf37a0d89d27 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:46.645355 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df07d760-220a-4051-92ce-cf37a0d89d27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:46.709024 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:33:46.732346 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-df07d760-220a-4051-92ce-cf37a0d89d27 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:33:46.737747 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-df07d760-220a-4051-92ce-cf37a0d89d27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:46.823790 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 956/1904] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:33:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:33:46.872494 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 949/1905] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:33:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:33:46.882330 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2882dc42-ed89-4bb0-8d5d-292a8fb75bfb None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:46.882620 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2882dc42-ed89-4bb0-8d5d-292a8fb75bfb None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:46.882958 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2882dc42-ed89-4bb0-8d5d-292a8fb75bfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:46.890702 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df07d760-220a-4051-92ce-cf37a0d89d27 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:46.894199 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-df07d760-220a-4051-92ce-cf37a0d89d27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:46.909842 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-2882dc42-ed89-4bb0-8d5d-292a8fb75bfb None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:33:46.911227 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-2882dc42-ed89-4bb0-8d5d-292a8fb75bfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:46.923343 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 957/1906] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:33:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:33:46.927246 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2882dc42-ed89-4bb0-8d5d-292a8fb75bfb None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:46.938567 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2882dc42-ed89-4bb0-8d5d-292a8fb75bfb None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:46.944802 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0072128f-d09a-4cc7-9bf6-4fd0dd191c58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:46.945483 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0072128f-d09a-4cc7-9bf6-4fd0dd191c58 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:46.945767 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0072128f-d09a-4cc7-9bf6-4fd0dd191c58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:46.946045 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0072128f-d09a-4cc7-9bf6-4fd0dd191c58 admin admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:46.947473 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0072128f-d09a-4cc7-9bf6-4fd0dd191c58 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:33:46.947473 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0072128f-d09a-4cc7-9bf6-4fd0dd191c58 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:46.948797 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0072128f-d09a-4cc7-9bf6-4fd0dd191c58 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:46.965188 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 958/1907] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Aug 6 01:33:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:33:46.966875 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0072128f-d09a-4cc7-9bf6-4fd0dd191c58 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:46.977671 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df07d760-220a-4051-92ce-cf37a0d89d27 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:46.978298 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0072128f-d09a-4cc7-9bf6-4fd0dd191c58 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:46.981499 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eea2f95-7909-49ae-bd76-1419787926eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:46.982198 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eea2f95-7909-49ae-bd76-1419787926eb service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:46.982556 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eea2f95-7909-49ae-bd76-1419787926eb service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:46.982911 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eea2f95-7909-49ae-bd76-1419787926eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:46.985088 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87881b56-ef80-4d13-a51b-db81f7af71d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:46.986949 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87881b56-ef80-4d13-a51b-db81f7af71d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:46.987310 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87881b56-ef80-4d13-a51b-db81f7af71d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:46.987801 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87881b56-ef80-4d13-a51b-db81f7af71d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:46.988543 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87881b56-ef80-4d13-a51b-db81f7af71d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:46.990859 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87881b56-ef80-4d13-a51b-db81f7af71d0 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:46.994291 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eea2f95-7909-49ae-bd76-1419787926eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:46.996123 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eea2f95-7909-49ae-bd76-1419787926eb service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:47.009681 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 950/1908] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:33:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 01:33:47.030789 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 959/1909] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Aug 6 01:33:46 2022] POST /identity/v3/projects => generated 401 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:33:47.031975 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eea2f95-7909-49ae-bd76-1419787926eb service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:47.043505 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eea2f95-7909-49ae-bd76-1419787926eb service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:47.135093 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec4dd30-d0ca-4723-96ad-81d2d47176b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:47.135785 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec4dd30-d0ca-4723-96ad-81d2d47176b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:47.136814 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec4dd30-d0ca-4723-96ad-81d2d47176b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:47.137140 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec4dd30-d0ca-4723-96ad-81d2d47176b1 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:47.138189 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec4dd30-d0ca-4723-96ad-81d2d47176b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:47.141627 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec4dd30-d0ca-4723-96ad-81d2d47176b1 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:47.183852 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 951/1910] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:33:47 2022] POST /identity/v3/users => generated 301 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:33:47.190686 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87881b56-ef80-4d13-a51b-db81f7af71d0 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:47.205184 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87881b56-ef80-4d13-a51b-db81f7af71d0 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:47.210932 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec32c3d1-5ee2-4dac-bd13-45769d5a5d89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:47.211568 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec32c3d1-5ee2-4dac-bd13-45769d5a5d89 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:47.211852 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec32c3d1-5ee2-4dac-bd13-45769d5a5d89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:47.212134 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec32c3d1-5ee2-4dac-bd13-45769d5a5d89 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:47.213008 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec32c3d1-5ee2-4dac-bd13-45769d5a5d89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:47.215044 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec32c3d1-5ee2-4dac-bd13-45769d5a5d89 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:47.227332 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 960/1911] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:33:47 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:33:47.231643 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec4dd30-d0ca-4723-96ad-81d2d47176b1 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:47.246356 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec4dd30-d0ca-4723-96ad-81d2d47176b1 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:47.251214 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20484934-3b2a-41cb-a6f4-665e79b47fe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:47.252665 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20484934-3b2a-41cb-a6f4-665e79b47fe9 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:47.252936 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20484934-3b2a-41cb-a6f4-665e79b47fe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:47.253252 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20484934-3b2a-41cb-a6f4-665e79b47fe9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:47.254806 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20484934-3b2a-41cb-a6f4-665e79b47fe9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:47.256944 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20484934-3b2a-41cb-a6f4-665e79b47fe9 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:47.273537 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 952/1912] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:33:47 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:33:47.274380 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec32c3d1-5ee2-4dac-bd13-45769d5a5d89 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:47.285897 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec32c3d1-5ee2-4dac-bd13-45769d5a5d89 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:47.291933 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f23d67b-202d-4abb-bad0-b1ccd26aef07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:47.292929 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f23d67b-202d-4abb-bad0-b1ccd26aef07 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:47.293452 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f23d67b-202d-4abb-bad0-b1ccd26aef07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:47.293866 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f23d67b-202d-4abb-bad0-b1ccd26aef07 admin admin] PATH_INFO: `/v3/projects/d14f8313f41e4f7790777013bff04780/users/0a5f1dd22e774118a8bc25f99280eb0f/roles/61b3b93d37a047e4bb8369b46fe20db9` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:47.312184 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f23d67b-202d-4abb-bad0-b1ccd26aef07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d14f8313f41e4f7790777013bff04780, user_id=0a5f1dd22e774118a8bc25f99280eb0f, role_id=61b3b93d37a047e4bb8369b46fe20db9)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:47.315830 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f23d67b-202d-4abb-bad0-b1ccd26aef07 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:47.334176 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 961/1913] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:33:47 2022] PUT /identity/v3/projects/d14f8313f41e4f7790777013bff04780/users/0a5f1dd22e774118a8bc25f99280eb0f/roles/61b3b93d37a047e4bb8369b46fe20db9 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:33:47.348785 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01677d6-2ae2-4d50-a2b3-090228bc18aa None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:47.348785 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01677d6-2ae2-4d50-a2b3-090228bc18aa None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:47.348785 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01677d6-2ae2-4d50-a2b3-090228bc18aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:47.350329 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f23d67b-202d-4abb-bad0-b1ccd26aef07 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:47.354671 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-7f23d67b-202d-4abb-bad0-b1ccd26aef07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:47.385699 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:33:47.397105 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-d01677d6-2ae2-4d50-a2b3-090228bc18aa None None] MFA Rules not processed for user `0a5f1dd22e774118a8bc25f99280eb0f`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:33:47.399876 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-d01677d6-2ae2-4d50-a2b3-090228bc18aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:47.441248 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f23d67b-202d-4abb-bad0-b1ccd26aef07 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:47.447002 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8c24f1-7197-4da9-a21d-b857d1cd9ad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:47.448317 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8c24f1-7197-4da9-a21d-b857d1cd9ad4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:47.448605 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8c24f1-7197-4da9-a21d-b857d1cd9ad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:47.448985 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8c24f1-7197-4da9-a21d-b857d1cd9ad4 admin admin] PATH_INFO: `/v3/projects/2a4663cc11bb4153bf690d36f499ce27` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:47.451347 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8c24f1-7197-4da9-a21d-b857d1cd9ad4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a4663cc11bb4153bf690d36f499ce27)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:47.453506 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8c24f1-7197-4da9-a21d-b857d1cd9ad4 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:47.487372 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 953/1914] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:33:47 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:33:47.492411 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01677d6-2ae2-4d50-a2b3-090228bc18aa None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:47.507997 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01677d6-2ae2-4d50-a2b3-090228bc18aa None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:47.516683 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:33:47.519888 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-2d8c24f1-7197-4da9-a21d-b857d1cd9ad4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a4663cc11bb4153bf690d36f499ce27', 'request_id': 'req-2d8c24f1-7197-4da9-a21d-b857d1cd9ad4'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:47.528517 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-2d8c24f1-7197-4da9-a21d-b857d1cd9ad4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a4663cc11bb4153bf690d36f499ce27', 'request_id': 'req-2d8c24f1-7197-4da9-a21d-b857d1cd9ad4'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:47.537453 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-2d8c24f1-7197-4da9-a21d-b857d1cd9ad4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a4663cc11bb4153bf690d36f499ce27', 'request_id': 'req-2d8c24f1-7197-4da9-a21d-b857d1cd9ad4'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:47.552305 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-2d8c24f1-7197-4da9-a21d-b857d1cd9ad4 admin admin] The token cache is being invalidate because project 2a4663cc11bb4153bf690d36f499ce27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:33:47.552674 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-2d8c24f1-7197-4da9-a21d-b857d1cd9ad4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:47.553470 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-2d8c24f1-7197-4da9-a21d-b857d1cd9ad4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:47.555386 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 962/1915] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Aug 6 01:33:47 2022] DELETE /identity/v3/projects/2a4663cc11bb4153bf690d36f499ce27 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:33:47.558943 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8c24f1-7197-4da9-a21d-b857d1cd9ad4 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:47.560619 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-2d8c24f1-7197-4da9-a21d-b857d1cd9ad4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:47.587332 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a72e4f-f42b-49ea-b70b-a1b8c916ccc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:47.587970 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a72e4f-f42b-49ea-b70b-a1b8c916ccc2 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:47.589785 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a72e4f-f42b-49ea-b70b-a1b8c916ccc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:47.590052 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a72e4f-f42b-49ea-b70b-a1b8c916ccc2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:47.590748 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a72e4f-f42b-49ea-b70b-a1b8c916ccc2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:47.592824 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a72e4f-f42b-49ea-b70b-a1b8c916ccc2 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:47.629957 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 954/1916] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Aug 6 01:33:47 2022] POST /identity/v3/projects => generated 401 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:33:47.634104 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a72e4f-f42b-49ea-b70b-a1b8c916ccc2 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:47.638302 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-c1a72e4f-f42b-49ea-b70b-a1b8c916ccc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:47.653394 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8c24f1-7197-4da9-a21d-b857d1cd9ad4 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:47.715798 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccd4b71-70ca-4bd5-bf51-34bb4c7b0088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:47.716539 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccd4b71-70ca-4bd5-bf51-34bb4c7b0088 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:47.716892 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccd4b71-70ca-4bd5-bf51-34bb4c7b0088 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:47.717282 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccd4b71-70ca-4bd5-bf51-34bb4c7b0088 admin admin] PATH_INFO: `/v3/users/c3eab80a79134e198ae472c31fa1764e` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:47.717704 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a72e4f-f42b-49ea-b70b-a1b8c916ccc2 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:47.718610 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccd4b71-70ca-4bd5-bf51-34bb4c7b0088 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3eab80a79134e198ae472c31fa1764e)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:47.722087 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccd4b71-70ca-4bd5-bf51-34bb4c7b0088 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:47.725230 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dfc26d-0f83-4ae0-a484-11ec1c61fbdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:47.725875 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dfc26d-0f83-4ae0-a484-11ec1c61fbdc admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:47.726158 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dfc26d-0f83-4ae0-a484-11ec1c61fbdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:47.726438 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dfc26d-0f83-4ae0-a484-11ec1c61fbdc admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:47.727120 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dfc26d-0f83-4ae0-a484-11ec1c61fbdc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:47.729112 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dfc26d-0f83-4ae0-a484-11ec1c61fbdc admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:47.753997 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 955/1917] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:33:47 2022] POST /identity/v3/users => generated 307 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:33:47.757980 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dfc26d-0f83-4ae0-a484-11ec1c61fbdc admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:47.771980 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dfc26d-0f83-4ae0-a484-11ec1c61fbdc admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:47.780704 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-6ccd4b71-70ca-4bd5-bf51-34bb4c7b0088 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3eab80a79134e198ae472c31fa1764e'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:47.786586 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1368470-6f2b-4cc7-99d0-064c3a4ac4b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:47.787443 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1368470-6f2b-4cc7-99d0-064c3a4ac4b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:47.787831 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1368470-6f2b-4cc7-99d0-064c3a4ac4b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:47.788774 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1368470-6f2b-4cc7-99d0-064c3a4ac4b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:47.789779 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1368470-6f2b-4cc7-99d0-064c3a4ac4b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:47.792448 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-6ccd4b71-70ca-4bd5-bf51-34bb4c7b0088 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3eab80a79134e198ae472c31fa1764e'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:47.792865 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1368470-6f2b-4cc7-99d0-064c3a4ac4b9 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:47.793395 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-6ccd4b71-70ca-4bd5-bf51-34bb4c7b0088 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3eab80a79134e198ae472c31fa1764e'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:47.794059 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-6ccd4b71-70ca-4bd5-bf51-34bb4c7b0088 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3eab80a79134e198ae472c31fa1764e'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:47.813581 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 956/1918] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:33:47 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 01:33:47.817913 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1368470-6f2b-4cc7-99d0-064c3a4ac4b9 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:47.819648 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-6ccd4b71-70ca-4bd5-bf51-34bb4c7b0088 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3eab80a79134e198ae472c31fa1764e'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:47.823612 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-e1368470-6f2b-4cc7-99d0-064c3a4ac4b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:47.833435 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:33:47.836370 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:33:47.839603 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 963/1919] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Aug 6 01:33:47 2022] DELETE /identity/v3/users/c3eab80a79134e198ae472c31fa1764e => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:33:47.845506 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1368470-6f2b-4cc7-99d0-064c3a4ac4b9 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:47.945764 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c285a11-6dcf-4f53-a70a-35e9078127a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:47.948284 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c285a11-6dcf-4f53-a70a-35e9078127a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:47.948724 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c285a11-6dcf-4f53-a70a-35e9078127a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:47.949258 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c285a11-6dcf-4f53-a70a-35e9078127a5 admin admin] PATH_INFO: `/v3/projects/31b6007f418541489fda970645517f2b/users/1bd73d75514246c5a3ffd842e0ad7c38/roles/4559b9a0837249f895a13c70096f8d38` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:47.971112 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c285a11-6dcf-4f53-a70a-35e9078127a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31b6007f418541489fda970645517f2b, user_id=1bd73d75514246c5a3ffd842e0ad7c38, role_id=4559b9a0837249f895a13c70096f8d38)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:47.974400 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c285a11-6dcf-4f53-a70a-35e9078127a5 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:47.994307 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 957/1920] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:33:47 2022] PUT /identity/v3/projects/31b6007f418541489fda970645517f2b/users/1bd73d75514246c5a3ffd842e0ad7c38/roles/4559b9a0837249f895a13c70096f8d38 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:33:47.998222 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccd4b71-70ca-4bd5-bf51-34bb4c7b0088 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:48.010157 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccd4b71-70ca-4bd5-bf51-34bb4c7b0088 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:48.082979 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbe529c-a6de-46d0-af52-7365a4fbf8ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:48.083812 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbe529c-a6de-46d0-af52-7365a4fbf8ea admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:48.084268 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbe529c-a6de-46d0-af52-7365a4fbf8ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:48.084686 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbe529c-a6de-46d0-af52-7365a4fbf8ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:48.085696 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbe529c-a6de-46d0-af52-7365a4fbf8ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:48.087944 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbe529c-a6de-46d0-af52-7365a4fbf8ea admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:48.104921 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 964/1921] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:33:47 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:33:48.109079 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c285a11-6dcf-4f53-a70a-35e9078127a5 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:48.120450 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c285a11-6dcf-4f53-a70a-35e9078127a5 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:48.125282 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc55ba92-dfd4-415c-bfd3-343f82421457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:48.125974 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc55ba92-dfd4-415c-bfd3-343f82421457 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:48.126265 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc55ba92-dfd4-415c-bfd3-343f82421457 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:48.126549 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc55ba92-dfd4-415c-bfd3-343f82421457 admin admin] PATH_INFO: `/v3/projects/31b6007f418541489fda970645517f2b/users/1bd73d75514246c5a3ffd842e0ad7c38/roles/4559b9a0837249f895a13c70096f8d38` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:48.129562 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc55ba92-dfd4-415c-bfd3-343f82421457 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31b6007f418541489fda970645517f2b, user_id=1bd73d75514246c5a3ffd842e0ad7c38, role_id=4559b9a0837249f895a13c70096f8d38)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:48.131922 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc55ba92-dfd4-415c-bfd3-343f82421457 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:48.148440 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 958/1922] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:33:48 2022] PUT /identity/v3/projects/31b6007f418541489fda970645517f2b/users/1bd73d75514246c5a3ffd842e0ad7c38/roles/4559b9a0837249f895a13c70096f8d38 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:33:48.161523 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5596991f-e2bd-488f-aba8-6b6936c343b3 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:48.161523 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5596991f-e2bd-488f-aba8-6b6936c343b3 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:48.161523 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5596991f-e2bd-488f-aba8-6b6936c343b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:48.164959 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc55ba92-dfd4-415c-bfd3-343f82421457 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:48.167250 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-cc55ba92-dfd4-415c-bfd3-343f82421457 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:48.195023 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-5596991f-e2bd-488f-aba8-6b6936c343b3 None None] MFA Rules not processed for user `1bd73d75514246c5a3ffd842e0ad7c38`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:33:48.196258 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-5596991f-e2bd-488f-aba8-6b6936c343b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:48.279923 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc55ba92-dfd4-415c-bfd3-343f82421457 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:48.289304 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 965/1923] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:33:48 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:33:48.289485 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa7e935-9271-42d1-b7cc-26211a42478f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:48.289485 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa7e935-9271-42d1-b7cc-26211a42478f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:48.289485 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa7e935-9271-42d1-b7cc-26211a42478f admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:48.289485 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa7e935-9271-42d1-b7cc-26211a42478f admin admin] PATH_INFO: `/v3/projects/a1a230f6ebed4710b351ba5d90821aaa` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:48.289809 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa7e935-9271-42d1-b7cc-26211a42478f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1a230f6ebed4710b351ba5d90821aaa)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:48.291046 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa7e935-9271-42d1-b7cc-26211a42478f admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:48.298304 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5596991f-e2bd-488f-aba8-6b6936c343b3 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:48.302019 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-5596991f-e2bd-488f-aba8-6b6936c343b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:48.356271 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-3aa7e935-9271-42d1-b7cc-26211a42478f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1a230f6ebed4710b351ba5d90821aaa', 'request_id': 'req-3aa7e935-9271-42d1-b7cc-26211a42478f'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:48.364499 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-3aa7e935-9271-42d1-b7cc-26211a42478f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1a230f6ebed4710b351ba5d90821aaa', 'request_id': 'req-3aa7e935-9271-42d1-b7cc-26211a42478f'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:48.371484 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-3aa7e935-9271-42d1-b7cc-26211a42478f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1a230f6ebed4710b351ba5d90821aaa', 'request_id': 'req-3aa7e935-9271-42d1-b7cc-26211a42478f'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:48.375366 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5596991f-e2bd-488f-aba8-6b6936c343b3 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:48.386484 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-3aa7e935-9271-42d1-b7cc-26211a42478f admin admin] The token cache is being invalidate because project a1a230f6ebed4710b351ba5d90821aaa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:33:48.386816 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-3aa7e935-9271-42d1-b7cc-26211a42478f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:48.387668 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-3aa7e935-9271-42d1-b7cc-26211a42478f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:48.389585 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 959/1924] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Aug 6 01:33:48 2022] DELETE /identity/v3/projects/a1a230f6ebed4710b351ba5d90821aaa => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:33:48.395851 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8797974-48a4-4bf3-bd3c-e5280401b625 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:48.396233 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8797974-48a4-4bf3-bd3c-e5280401b625 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:48.396522 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8797974-48a4-4bf3-bd3c-e5280401b625 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:48.413254 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:33:48.419182 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-e8797974-48a4-4bf3-bd3c-e5280401b625 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:33:48.420452 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-e8797974-48a4-4bf3-bd3c-e5280401b625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:48.421205 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:33:48.443069 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4772b9-9644-4231-8a80-d94a04f77053 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1be1e6173e374a60b9e1fbf51a1ab571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:48.443830 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4772b9-9644-4231-8a80-d94a04f77053 service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:48.444183 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4772b9-9644-4231-8a80-d94a04f77053 service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:48.444606 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4772b9-9644-4231-8a80-d94a04f77053 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:48.446476 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-4a4772b9-9644-4231-8a80-d94a04f77053 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:48.497513 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 960/1925] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:33:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:33:48.508219 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6e800f-7eaf-428d-8cac-34349f7e1afc None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:48.508507 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6e800f-7eaf-428d-8cac-34349f7e1afc None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:48.508795 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6e800f-7eaf-428d-8cac-34349f7e1afc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:48.528163 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4772b9-9644-4231-8a80-d94a04f77053 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:48.529686 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4772b9-9644-4231-8a80-d94a04f77053 service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:48.532399 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-4d6e800f-7eaf-428d-8cac-34349f7e1afc None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:33:48.535445 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-4d6e800f-7eaf-428d-8cac-34349f7e1afc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:48.541226 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 966/1926] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:33:48 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 01:33:48.549698 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 961/1927] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:33:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:33:48.551983 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4772b9-9644-4231-8a80-d94a04f77053 service cinder] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:48.563524 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4772b9-9644-4231-8a80-d94a04f77053 service cinder] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:48.569768 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23f3fff-f20a-4a53-8185-e9329a253af9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:48.570415 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23f3fff-f20a-4a53-8185-e9329a253af9 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:48.570728 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23f3fff-f20a-4a53-8185-e9329a253af9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:48.571015 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23f3fff-f20a-4a53-8185-e9329a253af9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:48.571719 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23f3fff-f20a-4a53-8185-e9329a253af9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:33:48.572030 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23f3fff-f20a-4a53-8185-e9329a253af9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:48.579278 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23f3fff-f20a-4a53-8185-e9329a253af9 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:48.587566 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 967/1928] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Aug 6 01:33:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:33:48.592249 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6e800f-7eaf-428d-8cac-34349f7e1afc None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:48.602650 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6e800f-7eaf-428d-8cac-34349f7e1afc None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:48.607926 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634ddb25-d40f-475f-ba97-b67826e5fb64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:48.608571 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634ddb25-d40f-475f-ba97-b67826e5fb64 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:48.608859 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634ddb25-d40f-475f-ba97-b67826e5fb64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:48.609181 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634ddb25-d40f-475f-ba97-b67826e5fb64 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:48.609882 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634ddb25-d40f-475f-ba97-b67826e5fb64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:48.611937 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634ddb25-d40f-475f-ba97-b67826e5fb64 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:48.652511 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23f3fff-f20a-4a53-8185-e9329a253af9 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:48.656581 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-b23f3fff-f20a-4a53-8185-e9329a253af9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:48.675698 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 962/1929] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Aug 6 01:33:48 2022] POST /identity/v3/projects => generated 371 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:33:48.676800 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634ddb25-d40f-475f-ba97-b67826e5fb64 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:48.677950 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23f3fff-f20a-4a53-8185-e9329a253af9 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:48.740478 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458bd547-c373-47e7-9e9c-1329eaa885ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1be1e6173e374a60b9e1fbf51a1ab571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:48.741401 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458bd547-c373-47e7-9e9c-1329eaa885ad service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:48.741811 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458bd547-c373-47e7-9e9c-1329eaa885ad service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:48.742237 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458bd547-c373-47e7-9e9c-1329eaa885ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:48.744293 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-458bd547-c373-47e7-9e9c-1329eaa885ad service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:48.772437 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634ddb25-d40f-475f-ba97-b67826e5fb64 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:48.778770 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83574eb-3b54-419d-9240-d7623be9afcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:48.779655 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83574eb-3b54-419d-9240-d7623be9afcd admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:48.779935 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83574eb-3b54-419d-9240-d7623be9afcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:48.780599 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83574eb-3b54-419d-9240-d7623be9afcd admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:48.780891 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83574eb-3b54-419d-9240-d7623be9afcd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:48.783210 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83574eb-3b54-419d-9240-d7623be9afcd admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:48.810194 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 963/1930] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:33:48 2022] POST /identity/v3/users => generated 286 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:33:48.814923 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83574eb-3b54-419d-9240-d7623be9afcd admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:48.819375 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458bd547-c373-47e7-9e9c-1329eaa885ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:48.820843 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458bd547-c373-47e7-9e9c-1329eaa885ad service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:48.825678 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83574eb-3b54-419d-9240-d7623be9afcd admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:48.830727 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17084724-e82e-4071-ae46-a11055e4829d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:48.831489 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17084724-e82e-4071-ae46-a11055e4829d admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:48.831597 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 968/1931] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:33:48 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:33:48.831782 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17084724-e82e-4071-ae46-a11055e4829d admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:48.832047 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17084724-e82e-4071-ae46-a11055e4829d admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:48.832747 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17084724-e82e-4071-ae46-a11055e4829d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:48.834387 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17084724-e82e-4071-ae46-a11055e4829d admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:48.849116 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 964/1932] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:33:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 01:33:48.851033 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458bd547-c373-47e7-9e9c-1329eaa885ad service cinder] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:48.873109 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458bd547-c373-47e7-9e9c-1329eaa885ad service cinder] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:48.879228 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1425b4-052d-4639-ac74-375d85129d48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:48.879985 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1425b4-052d-4639-ac74-375d85129d48 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:48.880368 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1425b4-052d-4639-ac74-375d85129d48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:48.880739 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1425b4-052d-4639-ac74-375d85129d48 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:48.883556 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1425b4-052d-4639-ac74-375d85129d48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:48.886161 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1425b4-052d-4639-ac74-375d85129d48 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:48.902455 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 969/1933] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:33:48 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:33:48.903060 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17084724-e82e-4071-ae46-a11055e4829d admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:48.932553 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17084724-e82e-4071-ae46-a11055e4829d admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:48.962653 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fc758b-9094-461f-b475-312fd47583a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:48.963624 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fc758b-9094-461f-b475-312fd47583a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:48.964008 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fc758b-9094-461f-b475-312fd47583a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:48.967388 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fc758b-9094-461f-b475-312fd47583a8 admin admin] PATH_INFO: `/v3/projects/dcec6bf872fb45b1a3b5d48e6df30f42/users/ef243999c13d4558b6ce2588c42a04f9/roles/61b3b93d37a047e4bb8369b46fe20db9` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:48.993729 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fc758b-9094-461f-b475-312fd47583a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dcec6bf872fb45b1a3b5d48e6df30f42, user_id=ef243999c13d4558b6ce2588c42a04f9, role_id=61b3b93d37a047e4bb8369b46fe20db9)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:48.997629 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fc758b-9094-461f-b475-312fd47583a8 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:49.026357 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 965/1934] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:33:48 2022] PUT /identity/v3/projects/dcec6bf872fb45b1a3b5d48e6df30f42/users/ef243999c13d4558b6ce2588c42a04f9/roles/61b3b93d37a047e4bb8369b46fe20db9 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:33:49.028614 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b7ed58-52a8-4335-a224-e720a5054ed4 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:49.029100 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b7ed58-52a8-4335-a224-e720a5054ed4 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:49.029537 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b7ed58-52a8-4335-a224-e720a5054ed4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:49.064300 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-63b7ed58-52a8-4335-a224-e720a5054ed4 None None] MFA Rules not processed for user `ef243999c13d4558b6ce2588c42a04f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:33:49.065971 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-63b7ed58-52a8-4335-a224-e720a5054ed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:49.073218 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:33:49.197179 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 970/1935] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:33:49 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:33:49.211682 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fc758b-9094-461f-b475-312fd47583a8 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:49.276420 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:33:49.303420 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fc758b-9094-461f-b475-312fd47583a8 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:49.311312 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4df048-cb54-44fa-87ee-6046a9235ea4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1be1e6173e374a60b9e1fbf51a1ab571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:49.312109 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4df048-cb54-44fa-87ee-6046a9235ea4 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:49.312496 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4df048-cb54-44fa-87ee-6046a9235ea4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:49.312874 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4df048-cb54-44fa-87ee-6046a9235ea4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:49.323598 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4df048-cb54-44fa-87ee-6046a9235ea4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:49.325124 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4df048-cb54-44fa-87ee-6046a9235ea4 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:49.337361 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 966/1936] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:33:49 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 01:33:49.358838 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b7ed58-52a8-4335-a224-e720a5054ed4 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:49.446844 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b7ed58-52a8-4335-a224-e720a5054ed4 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:49.451917 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28ae39f-59e8-4ab4-b803-6880553f0ff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:49.452671 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28ae39f-59e8-4ab4-b803-6880553f0ff0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:49.452955 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28ae39f-59e8-4ab4-b803-6880553f0ff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:49.453666 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28ae39f-59e8-4ab4-b803-6880553f0ff0 admin admin] PATH_INFO: `/v3/users/ef243999c13d4558b6ce2588c42a04f9` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:49.454633 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28ae39f-59e8-4ab4-b803-6880553f0ff0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef243999c13d4558b6ce2588c42a04f9)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:49.457625 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28ae39f-59e8-4ab4-b803-6880553f0ff0 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:49.504580 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-f28ae39f-59e8-4ab4-b803-6880553f0ff0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef243999c13d4558b6ce2588c42a04f9'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:49.511928 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-f28ae39f-59e8-4ab4-b803-6880553f0ff0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef243999c13d4558b6ce2588c42a04f9'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:49.512597 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-f28ae39f-59e8-4ab4-b803-6880553f0ff0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef243999c13d4558b6ce2588c42a04f9'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:49.513260 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-f28ae39f-59e8-4ab4-b803-6880553f0ff0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef243999c13d4558b6ce2588c42a04f9'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:49.525728 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-f28ae39f-59e8-4ab4-b803-6880553f0ff0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef243999c13d4558b6ce2588c42a04f9'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:49.541233 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 971/1937] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Aug 6 01:33:49 2022] DELETE /identity/v3/users/ef243999c13d4558b6ce2588c42a04f9 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:33:49.545945 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3e5882-4709-4ef1-b482-545837d2b7ee None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:49.546377 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3e5882-4709-4ef1-b482-545837d2b7ee None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:49.546732 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3e5882-4709-4ef1-b482-545837d2b7ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:49.565778 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-6d3e5882-4709-4ef1-b482-545837d2b7ee None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:33:49.566974 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-6d3e5882-4709-4ef1-b482-545837d2b7ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:49.628556 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 967/1938] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:33:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:33:49.638112 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28ae39f-59e8-4ab4-b803-6880553f0ff0 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:49.639508 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-f28ae39f-59e8-4ab4-b803-6880553f0ff0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:49.672821 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:33:49.702319 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28ae39f-59e8-4ab4-b803-6880553f0ff0 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:49.706595 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7fb417-8a14-4b06-826a-602b8e0ac4f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:49.707216 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7fb417-8a14-4b06-826a-602b8e0ac4f5 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:49.707523 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7fb417-8a14-4b06-826a-602b8e0ac4f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:49.707796 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7fb417-8a14-4b06-826a-602b8e0ac4f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:49.717202 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7fb417-8a14-4b06-826a-602b8e0ac4f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:49.718619 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7fb417-8a14-4b06-826a-602b8e0ac4f5 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:49.728386 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 972/1939] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:33:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:33:50.022731 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3e5882-4709-4ef1-b482-545837d2b7ee None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:50.024775 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-6d3e5882-4709-4ef1-b482-545837d2b7ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:50.037695 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3e5882-4709-4ef1-b482-545837d2b7ee None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:50.045920 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901133fc-7708-4c43-98b9-62a2f39688e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:50.046786 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901133fc-7708-4c43-98b9-62a2f39688e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:50.047132 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901133fc-7708-4c43-98b9-62a2f39688e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:50.047324 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901133fc-7708-4c43-98b9-62a2f39688e8 admin admin] PATH_INFO: `/v3/projects/dcec6bf872fb45b1a3b5d48e6df30f42` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:50.049833 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901133fc-7708-4c43-98b9-62a2f39688e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dcec6bf872fb45b1a3b5d48e6df30f42)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:50.051878 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901133fc-7708-4c43-98b9-62a2f39688e8 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:50.129746 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-901133fc-7708-4c43-98b9-62a2f39688e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dcec6bf872fb45b1a3b5d48e6df30f42', 'request_id': 'req-901133fc-7708-4c43-98b9-62a2f39688e8'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:50.136361 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-901133fc-7708-4c43-98b9-62a2f39688e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dcec6bf872fb45b1a3b5d48e6df30f42', 'request_id': 'req-901133fc-7708-4c43-98b9-62a2f39688e8'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:50.143565 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-901133fc-7708-4c43-98b9-62a2f39688e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dcec6bf872fb45b1a3b5d48e6df30f42', 'request_id': 'req-901133fc-7708-4c43-98b9-62a2f39688e8'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:50.174651 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-901133fc-7708-4c43-98b9-62a2f39688e8 admin admin] The token cache is being invalidate because project dcec6bf872fb45b1a3b5d48e6df30f42 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:33:50.174651 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-901133fc-7708-4c43-98b9-62a2f39688e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:50.174651 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-901133fc-7708-4c43-98b9-62a2f39688e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:50.174651 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 968/1940] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Aug 6 01:33:50 2022] DELETE /identity/v3/projects/dcec6bf872fb45b1a3b5d48e6df30f42 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:33:50.178918 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fffc6e-cbee-4f36-a140-23b19daa4db6 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:50.178918 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fffc6e-cbee-4f36-a140-23b19daa4db6 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:50.178918 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fffc6e-cbee-4f36-a140-23b19daa4db6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:50.200318 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-39fffc6e-cbee-4f36-a140-23b19daa4db6 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:33:50.202236 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-39fffc6e-cbee-4f36-a140-23b19daa4db6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:50.281094 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 973/1941] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:33:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:33:50.284074 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236da212-063b-4c54-9a94-c81615580c63 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:50.284581 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236da212-063b-4c54-9a94-c81615580c63 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:50.284944 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236da212-063b-4c54-9a94-c81615580c63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:50.306550 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-236da212-063b-4c54-9a94-c81615580c63 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:33:50.308090 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-236da212-063b-4c54-9a94-c81615580c63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:50.317028 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 969/1942] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:33:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:33:50.321494 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fffc6e-cbee-4f36-a140-23b19daa4db6 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:50.331628 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fffc6e-cbee-4f36-a140-23b19daa4db6 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:50.337355 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9b0bda-e9a4-49b6-b8fb-41d8e9050206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:50.338402 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9b0bda-e9a4-49b6-b8fb-41d8e9050206 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:50.338402 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9b0bda-e9a4-49b6-b8fb-41d8e9050206 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:50.338574 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9b0bda-e9a4-49b6-b8fb-41d8e9050206 admin admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:50.339224 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9b0bda-e9a4-49b6-b8fb-41d8e9050206 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:33:50.339535 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9b0bda-e9a4-49b6-b8fb-41d8e9050206 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:50.341344 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9b0bda-e9a4-49b6-b8fb-41d8e9050206 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:50.360044 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 974/1943] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Aug 6 01:33:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:33:50.365372 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236da212-063b-4c54-9a94-c81615580c63 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:50.376839 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236da212-063b-4c54-9a94-c81615580c63 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:50.383003 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883f6027-5994-43bb-b253-2d80646f4e8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:50.383879 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883f6027-5994-43bb-b253-2d80646f4e8a admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:50.384157 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883f6027-5994-43bb-b253-2d80646f4e8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:50.384434 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883f6027-5994-43bb-b253-2d80646f4e8a admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:50.385165 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883f6027-5994-43bb-b253-2d80646f4e8a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:50.387083 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883f6027-5994-43bb-b253-2d80646f4e8a admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:50.427977 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 970/1944] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Aug 6 01:33:50 2022] POST /identity/v3/projects => generated 371 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:33:50.429109 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9b0bda-e9a4-49b6-b8fb-41d8e9050206 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:50.440406 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9b0bda-e9a4-49b6-b8fb-41d8e9050206 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:50.500693 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba7847e-9dcc-4af7-9362-7121e6043626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:50.501344 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba7847e-9dcc-4af7-9362-7121e6043626 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:50.501656 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba7847e-9dcc-4af7-9362-7121e6043626 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:50.501935 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba7847e-9dcc-4af7-9362-7121e6043626 admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:50.502614 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba7847e-9dcc-4af7-9362-7121e6043626 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:50.504529 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba7847e-9dcc-4af7-9362-7121e6043626 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:50.527261 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 975/1945] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:33:50 2022] POST /identity/v3/users => generated 286 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:33:50.531415 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883f6027-5994-43bb-b253-2d80646f4e8a admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:50.543353 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883f6027-5994-43bb-b253-2d80646f4e8a admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:50.547886 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f19351-c3ad-4311-a434-f6cc84fcecfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:50.548619 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f19351-c3ad-4311-a434-f6cc84fcecfb admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:50.549467 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f19351-c3ad-4311-a434-f6cc84fcecfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:50.549743 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f19351-c3ad-4311-a434-f6cc84fcecfb admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:50.550649 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f19351-c3ad-4311-a434-f6cc84fcecfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:50.552290 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f19351-c3ad-4311-a434-f6cc84fcecfb admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:50.562471 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 971/1946] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:33:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:33:50.566855 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba7847e-9dcc-4af7-9362-7121e6043626 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:50.576767 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba7847e-9dcc-4af7-9362-7121e6043626 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:50.581944 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9bf9c1-4f36-44f9-b44b-43a2c87a17bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:50.582559 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9bf9c1-4f36-44f9-b44b-43a2c87a17bf admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:50.582846 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9bf9c1-4f36-44f9-b44b-43a2c87a17bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:50.583122 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9bf9c1-4f36-44f9-b44b-43a2c87a17bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:50.583844 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9bf9c1-4f36-44f9-b44b-43a2c87a17bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:50.585493 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9bf9c1-4f36-44f9-b44b-43a2c87a17bf admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:50.596900 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 976/1947] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:33:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:33:50.600992 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f19351-c3ad-4311-a434-f6cc84fcecfb admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:50.611752 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f19351-c3ad-4311-a434-f6cc84fcecfb admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:50.617170 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beb0d94-933f-4c26-ba09-eaabaf5324d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:50.617775 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beb0d94-933f-4c26-ba09-eaabaf5324d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:50.618048 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beb0d94-933f-4c26-ba09-eaabaf5324d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:50.618329 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beb0d94-933f-4c26-ba09-eaabaf5324d9 admin admin] PATH_INFO: `/v3/projects/fb5e4a84aaa141dab25f024eebf2a2b6/users/11060e2d93474e79ad13dcdb3379806a/roles/61b3b93d37a047e4bb8369b46fe20db9` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:50.633131 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2beb0d94-933f-4c26-ba09-eaabaf5324d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb5e4a84aaa141dab25f024eebf2a2b6, user_id=11060e2d93474e79ad13dcdb3379806a, role_id=61b3b93d37a047e4bb8369b46fe20db9)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:50.635781 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2beb0d94-933f-4c26-ba09-eaabaf5324d9 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:50.637956 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:33:50.650445 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 972/1948] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:33:50 2022] PUT /identity/v3/projects/fb5e4a84aaa141dab25f024eebf2a2b6/users/11060e2d93474e79ad13dcdb3379806a/roles/61b3b93d37a047e4bb8369b46fe20db9 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:33:50.662490 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a0c535-fa5d-4f98-a837-48eed3b31476 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:50.662784 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a0c535-fa5d-4f98-a837-48eed3b31476 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:50.663065 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a0c535-fa5d-4f98-a837-48eed3b31476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:50.692093 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-76a0c535-fa5d-4f98-a837-48eed3b31476 None None] MFA Rules not processed for user `11060e2d93474e79ad13dcdb3379806a`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:33:50.693340 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-76a0c535-fa5d-4f98-a837-48eed3b31476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:50.716444 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:33:50.759854 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 977/1949] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:33:50 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:33:50.771602 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beb0d94-933f-4c26-ba09-eaabaf5324d9 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:50.773349 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-2beb0d94-933f-4c26-ba09-eaabaf5324d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:50.829435 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beb0d94-933f-4c26-ba09-eaabaf5324d9 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:50.833747 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f66e57e-5de7-4721-8b6e-c9203fd9553d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1be1e6173e374a60b9e1fbf51a1ab571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:50.834372 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f66e57e-5de7-4721-8b6e-c9203fd9553d service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:50.834658 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f66e57e-5de7-4721-8b6e-c9203fd9553d service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:50.834936 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f66e57e-5de7-4721-8b6e-c9203fd9553d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:50.843439 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f66e57e-5de7-4721-8b6e-c9203fd9553d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:50.844871 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f66e57e-5de7-4721-8b6e-c9203fd9553d service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:50.854719 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 973/1950] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:33:50 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:33:50.881774 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a0c535-fa5d-4f98-a837-48eed3b31476 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:50.959201 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a0c535-fa5d-4f98-a837-48eed3b31476 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:50.975866 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46097924-d0bd-4168-8796-e2464edfa157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:50.976677 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46097924-d0bd-4168-8796-e2464edfa157 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:50.976963 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46097924-d0bd-4168-8796-e2464edfa157 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:50.977271 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46097924-d0bd-4168-8796-e2464edfa157 admin admin] PATH_INFO: `/v3/users/11060e2d93474e79ad13dcdb3379806a` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:50.978837 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46097924-d0bd-4168-8796-e2464edfa157 admin admin] RBAC: Authorizing `identity:delete_user(user_id=11060e2d93474e79ad13dcdb3379806a)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:50.980802 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46097924-d0bd-4168-8796-e2464edfa157 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:51.074894 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-46097924-d0bd-4168-8796-e2464edfa157 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11060e2d93474e79ad13dcdb3379806a'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:51.086836 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-46097924-d0bd-4168-8796-e2464edfa157 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11060e2d93474e79ad13dcdb3379806a'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:51.087621 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-46097924-d0bd-4168-8796-e2464edfa157 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11060e2d93474e79ad13dcdb3379806a'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:51.088508 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-46097924-d0bd-4168-8796-e2464edfa157 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11060e2d93474e79ad13dcdb3379806a'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:51.105991 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-46097924-d0bd-4168-8796-e2464edfa157 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11060e2d93474e79ad13dcdb3379806a'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:51.133533 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 978/1951] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Aug 6 01:33:50 2022] DELETE /identity/v3/users/11060e2d93474e79ad13dcdb3379806a => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:33:51.137996 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeedb90-43a6-4512-ae45-a09ebcce5da5 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:51.138619 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeedb90-43a6-4512-ae45-a09ebcce5da5 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:51.139115 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeedb90-43a6-4512-ae45-a09ebcce5da5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:51.166758 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-eeeedb90-43a6-4512-ae45-a09ebcce5da5 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:33:51.168068 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-eeeedb90-43a6-4512-ae45-a09ebcce5da5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:51.275476 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 974/1952] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:33:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:33:51.287313 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46097924-d0bd-4168-8796-e2464edfa157 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:51.288868 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-46097924-d0bd-4168-8796-e2464edfa157 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:51.355751 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:33:51.374079 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46097924-d0bd-4168-8796-e2464edfa157 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:51.379215 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecf3037-0fed-4ae2-b231-1a9b86dbeabd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:51.380917 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecf3037-0fed-4ae2-b231-1a9b86dbeabd service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:51.381336 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecf3037-0fed-4ae2-b231-1a9b86dbeabd service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:51.381710 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecf3037-0fed-4ae2-b231-1a9b86dbeabd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:51.404940 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecf3037-0fed-4ae2-b231-1a9b86dbeabd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:51.406613 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecf3037-0fed-4ae2-b231-1a9b86dbeabd service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:51.431593 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 979/1953] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:33:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:33:51.834548 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeedb90-43a6-4512-ae45-a09ebcce5da5 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:51.838128 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-eeeedb90-43a6-4512-ae45-a09ebcce5da5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:51.852239 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeedb90-43a6-4512-ae45-a09ebcce5da5 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:51.857657 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b96c23b-3a3d-4dc2-9d07-d0e1ca67802c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:51.858227 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b96c23b-3a3d-4dc2-9d07-d0e1ca67802c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:51.858534 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b96c23b-3a3d-4dc2-9d07-d0e1ca67802c admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:51.858776 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b96c23b-3a3d-4dc2-9d07-d0e1ca67802c admin admin] PATH_INFO: `/v3/projects/fb5e4a84aaa141dab25f024eebf2a2b6` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:51.860018 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b96c23b-3a3d-4dc2-9d07-d0e1ca67802c admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb5e4a84aaa141dab25f024eebf2a2b6)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:51.862537 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b96c23b-3a3d-4dc2-9d07-d0e1ca67802c admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:51.921891 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-2b96c23b-3a3d-4dc2-9d07-d0e1ca67802c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb5e4a84aaa141dab25f024eebf2a2b6', 'request_id': 'req-2b96c23b-3a3d-4dc2-9d07-d0e1ca67802c'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:51.929555 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-2b96c23b-3a3d-4dc2-9d07-d0e1ca67802c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb5e4a84aaa141dab25f024eebf2a2b6', 'request_id': 'req-2b96c23b-3a3d-4dc2-9d07-d0e1ca67802c'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:51.937111 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-2b96c23b-3a3d-4dc2-9d07-d0e1ca67802c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb5e4a84aaa141dab25f024eebf2a2b6', 'request_id': 'req-2b96c23b-3a3d-4dc2-9d07-d0e1ca67802c'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:51.953042 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-2b96c23b-3a3d-4dc2-9d07-d0e1ca67802c admin admin] The token cache is being invalidate because project fb5e4a84aaa141dab25f024eebf2a2b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:33:51.953636 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-2b96c23b-3a3d-4dc2-9d07-d0e1ca67802c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:51.954233 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-2b96c23b-3a3d-4dc2-9d07-d0e1ca67802c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:51.956671 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 975/1954] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Aug 6 01:33:51 2022] DELETE /identity/v3/projects/fb5e4a84aaa141dab25f024eebf2a2b6 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:33:51.961538 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bf1f98-326d-470b-9fac-776001911fc9 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:51.961830 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bf1f98-326d-470b-9fac-776001911fc9 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:51.962111 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bf1f98-326d-470b-9fac-776001911fc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:52.010538 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-38bf1f98-326d-470b-9fac-776001911fc9 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:33:52.012012 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-38bf1f98-326d-470b-9fac-776001911fc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:52.074657 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 980/1955] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:33:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:33:52.083154 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d76e890-5e42-41ce-89ee-deaaf6d49fb0 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:52.083438 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d76e890-5e42-41ce-89ee-deaaf6d49fb0 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:52.083717 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d76e890-5e42-41ce-89ee-deaaf6d49fb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:52.102939 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-8d76e890-5e42-41ce-89ee-deaaf6d49fb0 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:33:52.104146 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-8d76e890-5e42-41ce-89ee-deaaf6d49fb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:52.112451 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 976/1956] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:33:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:33:52.116876 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bf1f98-326d-470b-9fac-776001911fc9 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:52.126523 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bf1f98-326d-470b-9fac-776001911fc9 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:52.131775 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7866e85f-8399-4c57-870d-0e2a57b7ac27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:52.132324 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7866e85f-8399-4c57-870d-0e2a57b7ac27 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:52.132607 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7866e85f-8399-4c57-870d-0e2a57b7ac27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:52.132882 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7866e85f-8399-4c57-870d-0e2a57b7ac27 admin admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:52.133618 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7866e85f-8399-4c57-870d-0e2a57b7ac27 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:33:52.133926 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7866e85f-8399-4c57-870d-0e2a57b7ac27 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:52.135507 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7866e85f-8399-4c57-870d-0e2a57b7ac27 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:52.150615 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 981/1957] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Aug 6 01:33:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:33:52.154871 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d76e890-5e42-41ce-89ee-deaaf6d49fb0 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:52.164035 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d76e890-5e42-41ce-89ee-deaaf6d49fb0 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:52.168720 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c335ee-4546-439b-83c3-80a06fe7ce0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:52.169364 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c335ee-4546-439b-83c3-80a06fe7ce0d admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:52.169666 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c335ee-4546-439b-83c3-80a06fe7ce0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:52.169930 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c335ee-4546-439b-83c3-80a06fe7ce0d admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:52.170592 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c335ee-4546-439b-83c3-80a06fe7ce0d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:52.172503 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c335ee-4546-439b-83c3-80a06fe7ce0d admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:52.183693 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:33:52.201574 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 977/1958] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Aug 6 01:33:52 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:33:52.206565 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7866e85f-8399-4c57-870d-0e2a57b7ac27 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:52.216535 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7866e85f-8399-4c57-870d-0e2a57b7ac27 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:52.272853 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baa8075-1570-4ba6-a31c-46a80d1dab02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:52.273492 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baa8075-1570-4ba6-a31c-46a80d1dab02 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:52.273777 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baa8075-1570-4ba6-a31c-46a80d1dab02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:52.274054 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baa8075-1570-4ba6-a31c-46a80d1dab02 admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:52.274713 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baa8075-1570-4ba6-a31c-46a80d1dab02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:52.276635 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baa8075-1570-4ba6-a31c-46a80d1dab02 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:52.297468 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 982/1959] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:33:52 2022] POST /identity/v3/users => generated 292 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:33:52.301689 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c335ee-4546-439b-83c3-80a06fe7ce0d admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:52.311856 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c335ee-4546-439b-83c3-80a06fe7ce0d admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:52.316519 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0311db62-f647-481b-88d4-974b033942fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:52.317169 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0311db62-f647-481b-88d4-974b033942fc admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:52.317457 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0311db62-f647-481b-88d4-974b033942fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:52.317735 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0311db62-f647-481b-88d4-974b033942fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:52.318430 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0311db62-f647-481b-88d4-974b033942fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:52.320011 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0311db62-f647-481b-88d4-974b033942fc admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:52.330673 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 978/1960] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:33:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:33:52.334362 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baa8075-1570-4ba6-a31c-46a80d1dab02 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:52.346860 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baa8075-1570-4ba6-a31c-46a80d1dab02 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:52.351628 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f74ee8-484c-4667-8bcf-ef918fc31dd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:52.352245 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f74ee8-484c-4667-8bcf-ef918fc31dd0 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:52.352528 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f74ee8-484c-4667-8bcf-ef918fc31dd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:52.352801 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f74ee8-484c-4667-8bcf-ef918fc31dd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:52.353562 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f74ee8-484c-4667-8bcf-ef918fc31dd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:52.355159 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f74ee8-484c-4667-8bcf-ef918fc31dd0 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:52.365975 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 983/1961] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:33:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:33:52.370189 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0311db62-f647-481b-88d4-974b033942fc admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:52.380295 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0311db62-f647-481b-88d4-974b033942fc admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:52.385112 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eeed9a-6fdc-42c9-857b-5aeb118f5275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:52.385727 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eeed9a-6fdc-42c9-857b-5aeb118f5275 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:52.386006 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eeed9a-6fdc-42c9-857b-5aeb118f5275 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:52.386279 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eeed9a-6fdc-42c9-857b-5aeb118f5275 admin admin] PATH_INFO: `/v3/projects/e50641dc16fa4187b5f0d4f936d34d53/users/c938a575a2704078a82fe6df941884dc/roles/61b3b93d37a047e4bb8369b46fe20db9` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:52.401649 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5eeed9a-6fdc-42c9-857b-5aeb118f5275 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e50641dc16fa4187b5f0d4f936d34d53, user_id=c938a575a2704078a82fe6df941884dc, role_id=61b3b93d37a047e4bb8369b46fe20db9)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:52.404338 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5eeed9a-6fdc-42c9-857b-5aeb118f5275 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:52.418383 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 979/1962] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:33:52 2022] PUT /identity/v3/projects/e50641dc16fa4187b5f0d4f936d34d53/users/c938a575a2704078a82fe6df941884dc/roles/61b3b93d37a047e4bb8369b46fe20db9 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:33:52.423400 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7c5e18-7c98-4800-90dd-7dded65f17d5 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:52.424137 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7c5e18-7c98-4800-90dd-7dded65f17d5 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:52.424423 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7c5e18-7c98-4800-90dd-7dded65f17d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:52.443438 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-1e7c5e18-7c98-4800-90dd-7dded65f17d5 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:33:52.444621 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-1e7c5e18-7c98-4800-90dd-7dded65f17d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:52.493601 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:33:52.506628 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 984/1963] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:33:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:33:52.516237 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eeed9a-6fdc-42c9-857b-5aeb118f5275 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:52.518487 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-b5eeed9a-6fdc-42c9-857b-5aeb118f5275 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:52.578405 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eeed9a-6fdc-42c9-857b-5aeb118f5275 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:52.584136 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850681e8-f139-4329-97c1-22d2be1d611e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:52.584136 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850681e8-f139-4329-97c1-22d2be1d611e service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:52.584136 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850681e8-f139-4329-97c1-22d2be1d611e service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:52.589208 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850681e8-f139-4329-97c1-22d2be1d611e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:52.599847 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850681e8-f139-4329-97c1-22d2be1d611e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:52.602575 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850681e8-f139-4329-97c1-22d2be1d611e service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:52.613882 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 980/1964] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:33:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:33:53.624016 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68360895-4a3b-4b01-81a0-dd1991d9b0ca None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:53.624500 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68360895-4a3b-4b01-81a0-dd1991d9b0ca None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:53.624621 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68360895-4a3b-4b01-81a0-dd1991d9b0ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:53.655851 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-68360895-4a3b-4b01-81a0-dd1991d9b0ca None None] MFA Rules not processed for user `c938a575a2704078a82fe6df941884dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:33:53.657095 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-68360895-4a3b-4b01-81a0-dd1991d9b0ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:53.730051 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 985/1965] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:33:53 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:33:53.741859 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850681e8-f139-4329-97c1-22d2be1d611e service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:53.743380 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-850681e8-f139-4329-97c1-22d2be1d611e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:53.804295 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850681e8-f139-4329-97c1-22d2be1d611e service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:53.808012 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32199ddd-a50e-4b72-80ec-fd473f3d1cb2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1be1e6173e374a60b9e1fbf51a1ab571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:53.808630 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32199ddd-a50e-4b72-80ec-fd473f3d1cb2 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:53.808908 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32199ddd-a50e-4b72-80ec-fd473f3d1cb2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:53.809218 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32199ddd-a50e-4b72-80ec-fd473f3d1cb2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:53.817681 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32199ddd-a50e-4b72-80ec-fd473f3d1cb2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:53.819099 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32199ddd-a50e-4b72-80ec-fd473f3d1cb2 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:53.827738 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 981/1966] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:33:53 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:33:54.680145 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68360895-4a3b-4b01-81a0-dd1991d9b0ca None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:54.681892 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-68360895-4a3b-4b01-81a0-dd1991d9b0ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:54.702282 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68360895-4a3b-4b01-81a0-dd1991d9b0ca None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:54.712473 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c06ee9-4711-484b-8063-f1728f156299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:54.713166 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c06ee9-4711-484b-8063-f1728f156299 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:54.713575 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c06ee9-4711-484b-8063-f1728f156299 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:54.713945 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c06ee9-4711-484b-8063-f1728f156299 admin admin] PATH_INFO: `/v3/users/0a5f1dd22e774118a8bc25f99280eb0f` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:54.715527 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c06ee9-4711-484b-8063-f1728f156299 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a5f1dd22e774118a8bc25f99280eb0f)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:54.717700 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c06ee9-4711-484b-8063-f1728f156299 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:54.756841 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32199ddd-a50e-4b72-80ec-fd473f3d1cb2 service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:54.758951 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-32199ddd-a50e-4b72-80ec-fd473f3d1cb2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:54.778498 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32199ddd-a50e-4b72-80ec-fd473f3d1cb2 service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:54.780131 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-f8c06ee9-4711-484b-8063-f1728f156299 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a5f1dd22e774118a8bc25f99280eb0f'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:54.784036 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dbcbae-b9e2-471f-961e-30c0665b2852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:54.784743 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dbcbae-b9e2-471f-961e-30c0665b2852 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:54.785033 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dbcbae-b9e2-471f-961e-30c0665b2852 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:54.785339 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dbcbae-b9e2-471f-961e-30c0665b2852 admin admin] PATH_INFO: `/v3/users/3ebae341795649adba068af7f95c3992` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:54.786517 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dbcbae-b9e2-471f-961e-30c0665b2852 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ebae341795649adba068af7f95c3992)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:54.788245 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-f8c06ee9-4711-484b-8063-f1728f156299 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a5f1dd22e774118a8bc25f99280eb0f'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:54.788814 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dbcbae-b9e2-471f-961e-30c0665b2852 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:54.789975 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-f8c06ee9-4711-484b-8063-f1728f156299 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a5f1dd22e774118a8bc25f99280eb0f'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:54.790270 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-f8c06ee9-4711-484b-8063-f1728f156299 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a5f1dd22e774118a8bc25f99280eb0f'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:54.804980 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-f8c06ee9-4711-484b-8063-f1728f156299 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a5f1dd22e774118a8bc25f99280eb0f'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:54.824313 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 986/1967] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Aug 6 01:33:54 2022] DELETE /identity/v3/users/0a5f1dd22e774118a8bc25f99280eb0f => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:33:54.828702 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faff6731-4eb0-4e4f-8e61-beb64251d75c None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:54.828997 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faff6731-4eb0-4e4f-8e61-beb64251d75c None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:54.829309 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faff6731-4eb0-4e4f-8e61-beb64251d75c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:54.849870 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-faff6731-4eb0-4e4f-8e61-beb64251d75c None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:33:54.850644 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-22dbcbae-b9e2-471f-961e-30c0665b2852 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ebae341795649adba068af7f95c3992'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:54.851092 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-faff6731-4eb0-4e4f-8e61-beb64251d75c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:54.851484 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-22dbcbae-b9e2-471f-961e-30c0665b2852 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ebae341795649adba068af7f95c3992'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:54.852154 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-22dbcbae-b9e2-471f-961e-30c0665b2852 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ebae341795649adba068af7f95c3992'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:54.860176 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:33:54.874343 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-22dbcbae-b9e2-471f-961e-30c0665b2852 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ebae341795649adba068af7f95c3992'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:54.882836 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-22dbcbae-b9e2-471f-961e-30c0665b2852 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ebae341795649adba068af7f95c3992'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:54.898088 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 982/1968] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Aug 6 01:33:54 2022] DELETE /identity/v3/users/3ebae341795649adba068af7f95c3992 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:33:54.902567 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46ad170-54f2-402b-a083-3edcd5db5f0a None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:54.902926 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46ad170-54f2-402b-a083-3edcd5db5f0a None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:54.903280 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46ad170-54f2-402b-a083-3edcd5db5f0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:54.913268 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:33:54.924853 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-a46ad170-54f2-402b-a083-3edcd5db5f0a None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:33:54.926383 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-a46ad170-54f2-402b-a083-3edcd5db5f0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:54.932750 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 987/1969] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:33:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:33:54.939153 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faff6731-4eb0-4e4f-8e61-beb64251d75c None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:54.940508 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-faff6731-4eb0-4e4f-8e61-beb64251d75c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:54.999303 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 983/1970] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:33:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:33:55.003911 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faff6731-4eb0-4e4f-8e61-beb64251d75c None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:55.010959 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46ad170-54f2-402b-a083-3edcd5db5f0a None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:55.073786 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46ad170-54f2-402b-a083-3edcd5db5f0a None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:55.075424 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f19a27-1c66-4a2c-b983-89a8c7e99a12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:55.076063 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f19a27-1c66-4a2c-b983-89a8c7e99a12 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:55.076345 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f19a27-1c66-4a2c-b983-89a8c7e99a12 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:55.076632 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f19a27-1c66-4a2c-b983-89a8c7e99a12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:55.079366 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bacbe0-4086-431c-ad96-053dab389f17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:55.081619 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bacbe0-4086-431c-ad96-053dab389f17 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:55.082572 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bacbe0-4086-431c-ad96-053dab389f17 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:55.082572 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bacbe0-4086-431c-ad96-053dab389f17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:55.087897 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f19a27-1c66-4a2c-b983-89a8c7e99a12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:55.089380 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f19a27-1c66-4a2c-b983-89a8c7e99a12 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:55.093793 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bacbe0-4086-431c-ad96-053dab389f17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:55.097241 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bacbe0-4086-431c-ad96-053dab389f17 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:55.102181 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 988/1971] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:33:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 01:33:55.107777 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 984/1972] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:33:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 01:33:55.266807 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50994582-f489-492d-8ae6-3c5531e9f99c None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:55.269768 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50994582-f489-492d-8ae6-3c5531e9f99c None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:55.270941 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50994582-f489-492d-8ae6-3c5531e9f99c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:55.305687 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-50994582-f489-492d-8ae6-3c5531e9f99c None None] MFA Rules not processed for user `c938a575a2704078a82fe6df941884dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:33:55.306969 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-50994582-f489-492d-8ae6-3c5531e9f99c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:55.383669 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 989/1973] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:33:55 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:33:55.879538 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bacbe0-4086-431c-ad96-053dab389f17 service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:55.881167 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-16bacbe0-4086-431c-ad96-053dab389f17 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:55.897413 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bacbe0-4086-431c-ad96-053dab389f17 service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:55.902832 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac131108-7a62-4e8a-9f9a-b79ba9505769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:55.903961 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac131108-7a62-4e8a-9f9a-b79ba9505769 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:55.904401 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac131108-7a62-4e8a-9f9a-b79ba9505769 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:55.904833 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac131108-7a62-4e8a-9f9a-b79ba9505769 admin admin] PATH_INFO: `/v3/projects/d183e0a91de84b02ae7cd941c95a1040` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:55.906274 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac131108-7a62-4e8a-9f9a-b79ba9505769 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d183e0a91de84b02ae7cd941c95a1040)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:55.908628 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac131108-7a62-4e8a-9f9a-b79ba9505769 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:55.916103 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50994582-f489-492d-8ae6-3c5531e9f99c None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:55.917829 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-50994582-f489-492d-8ae6-3c5531e9f99c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:55.937529 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50994582-f489-492d-8ae6-3c5531e9f99c None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:55.944575 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937a217d-a249-490d-8062-bc4f19415427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:55.945940 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937a217d-a249-490d-8062-bc4f19415427 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:55.946332 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937a217d-a249-490d-8062-bc4f19415427 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:55.946805 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937a217d-a249-490d-8062-bc4f19415427 admin admin] PATH_INFO: `/v3/projects/d14f8313f41e4f7790777013bff04780` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:55.948518 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937a217d-a249-490d-8062-bc4f19415427 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d14f8313f41e4f7790777013bff04780)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:55.951361 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937a217d-a249-490d-8062-bc4f19415427 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:56.023485 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-937a217d-a249-490d-8062-bc4f19415427 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd14f8313f41e4f7790777013bff04780', 'request_id': 'req-937a217d-a249-490d-8062-bc4f19415427'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:56.028884 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-ac131108-7a62-4e8a-9f9a-b79ba9505769 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd183e0a91de84b02ae7cd941c95a1040', 'request_id': 'req-ac131108-7a62-4e8a-9f9a-b79ba9505769'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:56.030435 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-937a217d-a249-490d-8062-bc4f19415427 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd14f8313f41e4f7790777013bff04780', 'request_id': 'req-937a217d-a249-490d-8062-bc4f19415427'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:56.031882 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:33:56.036296 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-ac131108-7a62-4e8a-9f9a-b79ba9505769 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd183e0a91de84b02ae7cd941c95a1040', 'request_id': 'req-ac131108-7a62-4e8a-9f9a-b79ba9505769'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:56.038933 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-937a217d-a249-490d-8062-bc4f19415427 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd14f8313f41e4f7790777013bff04780', 'request_id': 'req-937a217d-a249-490d-8062-bc4f19415427'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:56.043203 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-ac131108-7a62-4e8a-9f9a-b79ba9505769 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd183e0a91de84b02ae7cd941c95a1040', 'request_id': 'req-ac131108-7a62-4e8a-9f9a-b79ba9505769'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:56.054973 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:33:56.056396 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-937a217d-a249-490d-8062-bc4f19415427 admin admin] The token cache is being invalidate because project d14f8313f41e4f7790777013bff04780 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:33:56.056645 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-937a217d-a249-490d-8062-bc4f19415427 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:56.057375 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-937a217d-a249-490d-8062-bc4f19415427 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:56.059253 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 990/1974] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Aug 6 01:33:55 2022] DELETE /identity/v3/projects/d14f8313f41e4f7790777013bff04780 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:33:56.060008 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-ac131108-7a62-4e8a-9f9a-b79ba9505769 admin admin] The token cache is being invalidate because project d183e0a91de84b02ae7cd941c95a1040 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:33:56.060479 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-ac131108-7a62-4e8a-9f9a-b79ba9505769 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:56.061762 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-ac131108-7a62-4e8a-9f9a-b79ba9505769 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:56.063170 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937a217d-a249-490d-8062-bc4f19415427 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:56.063801 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 985/1975] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Aug 6 01:33:55 2022] DELETE /identity/v3/projects/d183e0a91de84b02ae7cd941c95a1040 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:33:56.069094 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-937a217d-a249-490d-8062-bc4f19415427 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:56.069329 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac131108-7a62-4e8a-9f9a-b79ba9505769 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:56.070815 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-ac131108-7a62-4e8a-9f9a-b79ba9505769 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:56.150319 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937a217d-a249-490d-8062-bc4f19415427 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:56.154778 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac131108-7a62-4e8a-9f9a-b79ba9505769 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:56.158029 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39a635d-dff2-4e0d-8f22-282b53e286f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:56.159076 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39a635d-dff2-4e0d-8f22-282b53e286f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:56.159438 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39a635d-dff2-4e0d-8f22-282b53e286f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:56.159697 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39a635d-dff2-4e0d-8f22-282b53e286f1 admin admin] PATH_INFO: `/v3/users/1bd73d75514246c5a3ffd842e0ad7c38` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:56.161098 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39a635d-dff2-4e0d-8f22-282b53e286f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1bd73d75514246c5a3ffd842e0ad7c38)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:56.161695 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d42c0f6-aae4-4059-aae3-e08751804d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:56.162918 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d42c0f6-aae4-4059-aae3-e08751804d9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:56.162918 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d42c0f6-aae4-4059-aae3-e08751804d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:56.163106 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d42c0f6-aae4-4059-aae3-e08751804d9d admin admin] PATH_INFO: `/v3/users/560ae6277a21472a873c633f38374d5f` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:56.163209 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39a635d-dff2-4e0d-8f22-282b53e286f1 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:56.164367 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d42c0f6-aae4-4059-aae3-e08751804d9d admin admin] RBAC: Authorizing `identity:delete_user(user_id=560ae6277a21472a873c633f38374d5f)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:56.167418 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d42c0f6-aae4-4059-aae3-e08751804d9d admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:56.231760 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a39a635d-dff2-4e0d-8f22-282b53e286f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1bd73d75514246c5a3ffd842e0ad7c38'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:56.244352 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-2d42c0f6-aae4-4059-aae3-e08751804d9d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '560ae6277a21472a873c633f38374d5f'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:56.246118 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-2d42c0f6-aae4-4059-aae3-e08751804d9d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '560ae6277a21472a873c633f38374d5f'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:56.246721 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-2d42c0f6-aae4-4059-aae3-e08751804d9d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '560ae6277a21472a873c633f38374d5f'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:56.247332 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a39a635d-dff2-4e0d-8f22-282b53e286f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1bd73d75514246c5a3ffd842e0ad7c38'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:56.248932 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a39a635d-dff2-4e0d-8f22-282b53e286f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1bd73d75514246c5a3ffd842e0ad7c38'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:56.250589 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a39a635d-dff2-4e0d-8f22-282b53e286f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1bd73d75514246c5a3ffd842e0ad7c38'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:56.268087 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-2d42c0f6-aae4-4059-aae3-e08751804d9d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '560ae6277a21472a873c633f38374d5f'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:56.275944 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a39a635d-dff2-4e0d-8f22-282b53e286f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1bd73d75514246c5a3ffd842e0ad7c38'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:56.279341 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-2d42c0f6-aae4-4059-aae3-e08751804d9d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '560ae6277a21472a873c633f38374d5f'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:56.310256 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 991/1976] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Aug 6 01:33:56 2022] DELETE /identity/v3/users/1bd73d75514246c5a3ffd842e0ad7c38 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:33:56.310471 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 986/1977] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Aug 6 01:33:56 2022] DELETE /identity/v3/users/560ae6277a21472a873c633f38374d5f => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:33:57.308890 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39a635d-dff2-4e0d-8f22-282b53e286f1 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:57.311063 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-a39a635d-dff2-4e0d-8f22-282b53e286f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:57.332790 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d42c0f6-aae4-4059-aae3-e08751804d9d admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:57.334525 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-2d42c0f6-aae4-4059-aae3-e08751804d9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:57.452449 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39a635d-dff2-4e0d-8f22-282b53e286f1 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:57.471206 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:33:57.490671 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d42c0f6-aae4-4059-aae3-e08751804d9d admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:57.496203 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d71dd1a-bea5-4c09-a093-68d9c17fd9f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:57.496970 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d71dd1a-bea5-4c09-a093-68d9c17fd9f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:57.497361 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d71dd1a-bea5-4c09-a093-68d9c17fd9f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:57.497721 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d71dd1a-bea5-4c09-a093-68d9c17fd9f1 admin admin] PATH_INFO: `/v3/projects/31b6007f418541489fda970645517f2b` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:57.499683 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d71dd1a-bea5-4c09-a093-68d9c17fd9f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=31b6007f418541489fda970645517f2b)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:57.502150 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d71dd1a-bea5-4c09-a093-68d9c17fd9f1 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:57.548492 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:33:57.577455 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a143ea-ab08-4e50-b3a5-116a488e2e5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:57.578120 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a143ea-ab08-4e50-b3a5-116a488e2e5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:57.578394 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a143ea-ab08-4e50-b3a5-116a488e2e5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:57.578675 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a143ea-ab08-4e50-b3a5-116a488e2e5e admin admin] PATH_INFO: `/v3/projects/01bf7a59ecc645b5ab2268f4d2215525` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:57.583271 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a143ea-ab08-4e50-b3a5-116a488e2e5e admin admin] RBAC: Authorizing `identity:delete_project(project_id=01bf7a59ecc645b5ab2268f4d2215525)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:57.587668 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a143ea-ab08-4e50-b3a5-116a488e2e5e admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:57.600450 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-9d71dd1a-bea5-4c09-a093-68d9c17fd9f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31b6007f418541489fda970645517f2b', 'request_id': 'req-9d71dd1a-bea5-4c09-a093-68d9c17fd9f1'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:57.618670 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-9d71dd1a-bea5-4c09-a093-68d9c17fd9f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31b6007f418541489fda970645517f2b', 'request_id': 'req-9d71dd1a-bea5-4c09-a093-68d9c17fd9f1'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:57.630560 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-9d71dd1a-bea5-4c09-a093-68d9c17fd9f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31b6007f418541489fda970645517f2b', 'request_id': 'req-9d71dd1a-bea5-4c09-a093-68d9c17fd9f1'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:57.651536 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-9d71dd1a-bea5-4c09-a093-68d9c17fd9f1 admin admin] The token cache is being invalidate because project 31b6007f418541489fda970645517f2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:33:57.651773 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-9d71dd1a-bea5-4c09-a093-68d9c17fd9f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:57.652546 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-9d71dd1a-bea5-4c09-a093-68d9c17fd9f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:57.654541 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 987/1978] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Aug 6 01:33:57 2022] DELETE /identity/v3/projects/31b6007f418541489fda970645517f2b => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:33:57.660079 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3872762c-78a5-44e1-8b91-990a0af7c73b None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:57.660368 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3872762c-78a5-44e1-8b91-990a0af7c73b None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:57.660679 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3872762c-78a5-44e1-8b91-990a0af7c73b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:57.662133 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-90a143ea-ab08-4e50-b3a5-116a488e2e5e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01bf7a59ecc645b5ab2268f4d2215525', 'request_id': 'req-90a143ea-ab08-4e50-b3a5-116a488e2e5e'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:57.670077 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-90a143ea-ab08-4e50-b3a5-116a488e2e5e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01bf7a59ecc645b5ab2268f4d2215525', 'request_id': 'req-90a143ea-ab08-4e50-b3a5-116a488e2e5e'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:57.682564 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-90a143ea-ab08-4e50-b3a5-116a488e2e5e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01bf7a59ecc645b5ab2268f4d2215525', 'request_id': 'req-90a143ea-ab08-4e50-b3a5-116a488e2e5e'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:57.691331 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-3872762c-78a5-44e1-8b91-990a0af7c73b None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:33:57.692911 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-3872762c-78a5-44e1-8b91-990a0af7c73b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:57.703800 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-90a143ea-ab08-4e50-b3a5-116a488e2e5e admin admin] The token cache is being invalidate because project 01bf7a59ecc645b5ab2268f4d2215525 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:33:57.704099 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-90a143ea-ab08-4e50-b3a5-116a488e2e5e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:57.705328 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-90a143ea-ab08-4e50-b3a5-116a488e2e5e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:57.707319 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 992/1979] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Aug 6 01:33:57 2022] DELETE /identity/v3/projects/01bf7a59ecc645b5ab2268f4d2215525 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:33:57.712211 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a8cb39-eae3-46a6-a18d-fb0f40a3777c None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:57.712506 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a8cb39-eae3-46a6-a18d-fb0f40a3777c None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:57.712803 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a8cb39-eae3-46a6-a18d-fb0f40a3777c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:57.743074 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-b6a8cb39-eae3-46a6-a18d-fb0f40a3777c None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:33:57.755482 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-b6a8cb39-eae3-46a6-a18d-fb0f40a3777c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:57.813191 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 988/1980] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:33:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:33:57.817506 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2136e2-918c-4fff-acea-90d8236b8a3a None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:57.818835 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2136e2-918c-4fff-acea-90d8236b8a3a None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:57.822526 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2136e2-918c-4fff-acea-90d8236b8a3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:57.838275 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 993/1981] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:33:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:33:57.847087 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2f438d-ce02-4587-8fbd-8590723116c2 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:57.847389 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2f438d-ce02-4587-8fbd-8590723116c2 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:57.847702 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2f438d-ce02-4587-8fbd-8590723116c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:57.870120 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-4f2f438d-ce02-4587-8fbd-8590723116c2 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:33:57.871463 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-4f2f438d-ce02-4587-8fbd-8590723116c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:57.874977 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-0b2136e2-918c-4fff-acea-90d8236b8a3a None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:33:57.876232 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-0b2136e2-918c-4fff-acea-90d8236b8a3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:57.884471 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 994/1982] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:33:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:33:57.887404 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2f438d-ce02-4587-8fbd-8590723116c2 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:57.889942 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 989/1983] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:33:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:33:57.894734 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2136e2-918c-4fff-acea-90d8236b8a3a None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:57.910671 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2f438d-ce02-4587-8fbd-8590723116c2 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:57.914074 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2136e2-918c-4fff-acea-90d8236b8a3a None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:57.916758 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ea69e6-1552-4489-bb3a-79742edfa20a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:57.917585 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ea69e6-1552-4489-bb3a-79742edfa20a admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:57.919187 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ea69e6-1552-4489-bb3a-79742edfa20a admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:57.919187 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ea69e6-1552-4489-bb3a-79742edfa20a admin admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:57.919187 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ea69e6-1552-4489-bb3a-79742edfa20a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:33:57.919443 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ea69e6-1552-4489-bb3a-79742edfa20a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:57.921398 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ea69e6-1552-4489-bb3a-79742edfa20a admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:57.929438 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7efec7e-1333-411b-8506-c7e2f394f033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:57.931578 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7efec7e-1333-411b-8506-c7e2f394f033 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:57.931889 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7efec7e-1333-411b-8506-c7e2f394f033 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:57.933022 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7efec7e-1333-411b-8506-c7e2f394f033 admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:57.939520 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7efec7e-1333-411b-8506-c7e2f394f033 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:33:57.939696 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 995/1984] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Aug 6 01:33:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:33:57.939802 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7efec7e-1333-411b-8506-c7e2f394f033 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:57.941154 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ea69e6-1552-4489-bb3a-79742edfa20a admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:57.941770 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7efec7e-1333-411b-8506-c7e2f394f033 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:57.957464 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ea69e6-1552-4489-bb3a-79742edfa20a admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:57.958246 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 990/1985] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Aug 6 01:33:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:33:57.962808 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e6266c-c1e7-4ba8-a275-6f8b88ed87f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:57.963612 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e6266c-c1e7-4ba8-a275-6f8b88ed87f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:57.963867 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7efec7e-1333-411b-8506-c7e2f394f033 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:57.964003 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e6266c-c1e7-4ba8-a275-6f8b88ed87f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:57.964210 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e6266c-c1e7-4ba8-a275-6f8b88ed87f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:57.965008 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e6266c-c1e7-4ba8-a275-6f8b88ed87f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:57.967039 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e6266c-c1e7-4ba8-a275-6f8b88ed87f9 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:57.977602 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7efec7e-1333-411b-8506-c7e2f394f033 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:57.982760 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983fbb56-ce2e-40e5-b83f-57c401078446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:57.984081 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983fbb56-ce2e-40e5-b83f-57c401078446 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:57.984371 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983fbb56-ce2e-40e5-b83f-57c401078446 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:57.984648 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983fbb56-ce2e-40e5-b83f-57c401078446 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:57.985404 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983fbb56-ce2e-40e5-b83f-57c401078446 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:57.987992 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983fbb56-ce2e-40e5-b83f-57c401078446 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:57.993487 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:33:58.004634 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 996/1986] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Aug 6 01:33:57 2022] POST /identity/v3/projects => generated 367 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:33:58.008693 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e6266c-c1e7-4ba8-a275-6f8b88ed87f9 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:58.037320 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 991/1987] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Aug 6 01:33:57 2022] POST /identity/v3/projects => generated 379 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:33:58.038248 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983fbb56-ce2e-40e5-b83f-57c401078446 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:58.083924 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e6266c-c1e7-4ba8-a275-6f8b88ed87f9 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:58.120135 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983fbb56-ce2e-40e5-b83f-57c401078446 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:58.126082 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ea7067-9197-4dd0-8cfe-b6cd7a01f30f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:58.127438 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ea7067-9197-4dd0-8cfe-b6cd7a01f30f admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:58.127762 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ea7067-9197-4dd0-8cfe-b6cd7a01f30f admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:58.128175 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ea7067-9197-4dd0-8cfe-b6cd7a01f30f admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:58.128860 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ea7067-9197-4dd0-8cfe-b6cd7a01f30f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:58.130972 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ea7067-9197-4dd0-8cfe-b6cd7a01f30f admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:58.151719 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5095d615-f664-48f1-9a7d-5d9df6fe0cda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:58.151948 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:DELETE=7,keystone:SELECT=88,keystone:INSERT=5 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:33:58.152540 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5095d615-f664-48f1-9a7d-5d9df6fe0cda admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:58.152678 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5095d615-f664-48f1-9a7d-5d9df6fe0cda admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:58.153009 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5095d615-f664-48f1-9a7d-5d9df6fe0cda admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:58.153725 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5095d615-f664-48f1-9a7d-5d9df6fe0cda admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:58.155929 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5095d615-f664-48f1-9a7d-5d9df6fe0cda admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:58.166251 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 992/1988] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:33:58 2022] POST /identity/v3/users => generated 290 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:33:58.168069 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ea7067-9197-4dd0-8cfe-b6cd7a01f30f admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:58.181236 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ea7067-9197-4dd0-8cfe-b6cd7a01f30f admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:58.187280 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03cdf97-ee25-4df1-943c-7e1585029f08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:58.187959 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03cdf97-ee25-4df1-943c-7e1585029f08 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:58.188247 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03cdf97-ee25-4df1-943c-7e1585029f08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:58.188529 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03cdf97-ee25-4df1-943c-7e1585029f08 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:58.189844 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03cdf97-ee25-4df1-943c-7e1585029f08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:58.191690 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03cdf97-ee25-4df1-943c-7e1585029f08 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:58.193387 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 997/1989] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:33:58 2022] POST /identity/v3/users => generated 284 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:33:58.197363 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5095d615-f664-48f1-9a7d-5d9df6fe0cda admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:58.207014 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 993/1990] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:33:58 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:33:58.211105 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03cdf97-ee25-4df1-943c-7e1585029f08 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:58.211328 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5095d615-f664-48f1-9a7d-5d9df6fe0cda admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:58.216072 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbb32ce-94be-48b9-ba4d-2ddafff17f03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:58.216715 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbb32ce-94be-48b9-ba4d-2ddafff17f03 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:58.217000 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbb32ce-94be-48b9-ba4d-2ddafff17f03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:58.217342 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbb32ce-94be-48b9-ba4d-2ddafff17f03 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:58.218041 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbb32ce-94be-48b9-ba4d-2ddafff17f03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:58.219732 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbb32ce-94be-48b9-ba4d-2ddafff17f03 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:58.222030 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03cdf97-ee25-4df1-943c-7e1585029f08 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:58.227775 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda044f6-247f-43b6-b54a-fd93f18a89eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:58.228394 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda044f6-247f-43b6-b54a-fd93f18a89eb admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:58.228812 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda044f6-247f-43b6-b54a-fd93f18a89eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:58.229123 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda044f6-247f-43b6-b54a-fd93f18a89eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:58.229818 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda044f6-247f-43b6-b54a-fd93f18a89eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:58.231458 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 998/1991] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:33:58 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 01:33:58.231577 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda044f6-247f-43b6-b54a-fd93f18a89eb admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:58.235338 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbb32ce-94be-48b9-ba4d-2ddafff17f03 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:58.249449 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 994/1992] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:33:58 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 01:33:58.249645 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbb32ce-94be-48b9-ba4d-2ddafff17f03 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:58.250907 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda044f6-247f-43b6-b54a-fd93f18a89eb admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:58.252673 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28a3994-58a8-4838-918b-0598ada3db76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:58.254030 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28a3994-58a8-4838-918b-0598ada3db76 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:58.254030 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28a3994-58a8-4838-918b-0598ada3db76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:58.254227 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28a3994-58a8-4838-918b-0598ada3db76 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:58.254847 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28a3994-58a8-4838-918b-0598ada3db76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:58.256444 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28a3994-58a8-4838-918b-0598ada3db76 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:58.261985 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda044f6-247f-43b6-b54a-fd93f18a89eb admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:58.266933 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0598fb7-e41b-404d-9908-5f6fae5dcb40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:58.267972 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0598fb7-e41b-404d-9908-5f6fae5dcb40 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:58.268174 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0598fb7-e41b-404d-9908-5f6fae5dcb40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:58.271466 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0598fb7-e41b-404d-9908-5f6fae5dcb40 admin admin] PATH_INFO: `/v3/projects/044dd2763cda4ffeb3518f0d622425c4/users/29cd637c61d7444bbfca52b13342b801/roles/61b3b93d37a047e4bb8369b46fe20db9` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:58.271585 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 999/1993] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:33:58 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 01:33:58.272522 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28a3994-58a8-4838-918b-0598ada3db76 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:58.282566 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28a3994-58a8-4838-918b-0598ada3db76 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:58.287554 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ebeb46-659e-48cf-a2e5-4328e0e4df16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:58.288219 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ebeb46-659e-48cf-a2e5-4328e0e4df16 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:58.288500 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ebeb46-659e-48cf-a2e5-4328e0e4df16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:58.288775 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ebeb46-659e-48cf-a2e5-4328e0e4df16 admin admin] PATH_INFO: `/v3/projects/34d1896282a044b4a37fcd56cf14b14e/users/d77a4cd10b264ddeac77994f0d690f29/roles/61b3b93d37a047e4bb8369b46fe20db9` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:58.289517 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0598fb7-e41b-404d-9908-5f6fae5dcb40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=044dd2763cda4ffeb3518f0d622425c4, user_id=29cd637c61d7444bbfca52b13342b801, role_id=61b3b93d37a047e4bb8369b46fe20db9)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:58.293025 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0598fb7-e41b-404d-9908-5f6fae5dcb40 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:58.305873 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ebeb46-659e-48cf-a2e5-4328e0e4df16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34d1896282a044b4a37fcd56cf14b14e, user_id=d77a4cd10b264ddeac77994f0d690f29, role_id=61b3b93d37a047e4bb8369b46fe20db9)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:58.308600 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ebeb46-659e-48cf-a2e5-4328e0e4df16 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:58.311203 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 995/1994] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:33:58 2022] PUT /identity/v3/projects/044dd2763cda4ffeb3518f0d622425c4/users/29cd637c61d7444bbfca52b13342b801/roles/61b3b93d37a047e4bb8369b46fe20db9 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:33:58.323332 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d50dbdf-eb5d-4d63-8f9d-b531f5093a67 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:58.332719 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1000/1995] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:33:58 2022] PUT /identity/v3/projects/34d1896282a044b4a37fcd56cf14b14e/users/d77a4cd10b264ddeac77994f0d690f29/roles/61b3b93d37a047e4bb8369b46fe20db9 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:33:58.334969 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d50dbdf-eb5d-4d63-8f9d-b531f5093a67 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:58.334969 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d50dbdf-eb5d-4d63-8f9d-b531f5093a67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:58.342722 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661cccf1-7686-4b18-a03e-642fe3a307e3 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:58.343143 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661cccf1-7686-4b18-a03e-642fe3a307e3 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:58.343464 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661cccf1-7686-4b18-a03e-642fe3a307e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:58.369485 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-3d50dbdf-eb5d-4d63-8f9d-b531f5093a67 None None] MFA Rules not processed for user `29cd637c61d7444bbfca52b13342b801`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:33:58.370742 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-3d50dbdf-eb5d-4d63-8f9d-b531f5093a67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:58.388220 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-661cccf1-7686-4b18-a03e-642fe3a307e3 None None] MFA Rules not processed for user `d77a4cd10b264ddeac77994f0d690f29`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:33:58.389574 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-661cccf1-7686-4b18-a03e-642fe3a307e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:58.433181 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:33:58.481721 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 996/1996] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:33:58 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:33:58.481721 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d50dbdf-eb5d-4d63-8f9d-b531f5093a67 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:58.484211 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1001/1997] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:33:58 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:33:58.488571 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661cccf1-7686-4b18-a03e-642fe3a307e3 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:58.560228 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d50dbdf-eb5d-4d63-8f9d-b531f5093a67 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:58.561848 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661cccf1-7686-4b18-a03e-642fe3a307e3 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:58.566187 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fec0f7-5e24-4d24-a462-0c33b5e1ae2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:58.567184 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fec0f7-5e24-4d24-a462-0c33b5e1ae2d admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:58.567295 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1370fabd-f750-434b-a7fb-8322623e8cf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:58.567774 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fec0f7-5e24-4d24-a462-0c33b5e1ae2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:58.567873 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1370fabd-f750-434b-a7fb-8322623e8cf5 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:58.568130 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1370fabd-f750-434b-a7fb-8322623e8cf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:58.568360 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fec0f7-5e24-4d24-a462-0c33b5e1ae2d admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:58.568461 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1370fabd-f750-434b-a7fb-8322623e8cf5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:58.569073 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1370fabd-f750-434b-a7fb-8322623e8cf5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:58.569618 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fec0f7-5e24-4d24-a462-0c33b5e1ae2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:58.571012 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1370fabd-f750-434b-a7fb-8322623e8cf5 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:58.571945 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fec0f7-5e24-4d24-a462-0c33b5e1ae2d admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:58.603051 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1002/1998] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Aug 6 01:33:58 2022] POST /identity/v3/projects => generated 367 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:33:58.603889 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 997/1999] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Aug 6 01:33:58 2022] POST /identity/v3/projects => generated 379 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:33:58.604192 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1370fabd-f750-434b-a7fb-8322623e8cf5 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:58.609301 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fec0f7-5e24-4d24-a462-0c33b5e1ae2d admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:58.645215 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:33:58.684918 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1370fabd-f750-434b-a7fb-8322623e8cf5 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:58.691082 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459d5f88-ee54-474e-90ae-c9cbf3c5e2f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:58.691726 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459d5f88-ee54-474e-90ae-c9cbf3c5e2f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:58.692009 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459d5f88-ee54-474e-90ae-c9cbf3c5e2f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:58.692326 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459d5f88-ee54-474e-90ae-c9cbf3c5e2f8 admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:58.694885 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459d5f88-ee54-474e-90ae-c9cbf3c5e2f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:58.695044 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459d5f88-ee54-474e-90ae-c9cbf3c5e2f8 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:58.697656 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fec0f7-5e24-4d24-a462-0c33b5e1ae2d admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:58.702718 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ce8733-88de-49f3-8878-bf703ed81ee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:58.703575 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ce8733-88de-49f3-8878-bf703ed81ee9 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:58.703860 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ce8733-88de-49f3-8878-bf703ed81ee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:58.704140 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ce8733-88de-49f3-8878-bf703ed81ee9 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:58.704959 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ce8733-88de-49f3-8878-bf703ed81ee9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:58.706967 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ce8733-88de-49f3-8878-bf703ed81ee9 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:58.722755 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1003/2000] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:33:58 2022] POST /identity/v3/users => generated 290 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:33:58.726881 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459d5f88-ee54-474e-90ae-c9cbf3c5e2f8 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:58.733919 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 998/2001] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:33:58 2022] POST /identity/v3/users => generated 296 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:33:58.739131 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ce8733-88de-49f3-8878-bf703ed81ee9 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:58.740090 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459d5f88-ee54-474e-90ae-c9cbf3c5e2f8 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:58.746522 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68939cd8-f3f2-4ad3-87ae-ecf576af3a32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:58.747160 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68939cd8-f3f2-4ad3-87ae-ecf576af3a32 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:58.747452 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68939cd8-f3f2-4ad3-87ae-ecf576af3a32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:58.747899 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68939cd8-f3f2-4ad3-87ae-ecf576af3a32 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:58.748582 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68939cd8-f3f2-4ad3-87ae-ecf576af3a32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:58.750295 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68939cd8-f3f2-4ad3-87ae-ecf576af3a32 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:58.750412 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ce8733-88de-49f3-8878-bf703ed81ee9 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:58.756294 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79a7b17-33ec-459b-9d66-ec30800e5b6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:58.763941 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1004/2002] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:33:58 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 01:33:58.769200 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68939cd8-f3f2-4ad3-87ae-ecf576af3a32 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:58.770539 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79a7b17-33ec-459b-9d66-ec30800e5b6e admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:58.770833 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79a7b17-33ec-459b-9d66-ec30800e5b6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:58.771112 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79a7b17-33ec-459b-9d66-ec30800e5b6e admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:58.771840 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79a7b17-33ec-459b-9d66-ec30800e5b6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:58.773494 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79a7b17-33ec-459b-9d66-ec30800e5b6e admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:58.782139 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68939cd8-f3f2-4ad3-87ae-ecf576af3a32 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:58.784609 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 999/2003] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:33:58 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 01:33:58.788410 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79a7b17-33ec-459b-9d66-ec30800e5b6e admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:58.792640 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace337d6-c090-4a3b-a988-4fea338d7347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:58.793374 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace337d6-c090-4a3b-a988-4fea338d7347 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:58.793860 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace337d6-c090-4a3b-a988-4fea338d7347 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:58.794144 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace337d6-c090-4a3b-a988-4fea338d7347 admin admin] PATH_INFO: `/v3/projects/2fabcd4bf2fc4e2f8ed4f28302348d24/users/4e919a58ca13432a90fe8367a4b7493d/roles/4559b9a0837249f895a13c70096f8d38` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:58.800354 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79a7b17-33ec-459b-9d66-ec30800e5b6e admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:58.806847 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b549f211-1392-490c-b3b3-20f5d396c33f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:58.808221 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b549f211-1392-490c-b3b3-20f5d396c33f admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:58.808626 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b549f211-1392-490c-b3b3-20f5d396c33f admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:58.809033 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b549f211-1392-490c-b3b3-20f5d396c33f admin admin] PATH_INFO: `/v3/projects/a1b30dc6b8df42459be65cb8c1e4b27a/users/824d848767534057989e1fb0830b3a83/roles/4559b9a0837249f895a13c70096f8d38` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:58.830994 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b549f211-1392-490c-b3b3-20f5d396c33f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1b30dc6b8df42459be65cb8c1e4b27a, user_id=824d848767534057989e1fb0830b3a83, role_id=4559b9a0837249f895a13c70096f8d38)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:58.834371 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b549f211-1392-490c-b3b3-20f5d396c33f admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:58.839407 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace337d6-c090-4a3b-a988-4fea338d7347 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fabcd4bf2fc4e2f8ed4f28302348d24, user_id=4e919a58ca13432a90fe8367a4b7493d, role_id=4559b9a0837249f895a13c70096f8d38)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:58.842787 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace337d6-c090-4a3b-a988-4fea338d7347 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:58.850939 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1000/2004] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:33:58 2022] PUT /identity/v3/projects/a1b30dc6b8df42459be65cb8c1e4b27a/users/824d848767534057989e1fb0830b3a83/roles/4559b9a0837249f895a13c70096f8d38 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:33:58.855333 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b549f211-1392-490c-b3b3-20f5d396c33f admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:58.870200 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1005/2005] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:33:58 2022] PUT /identity/v3/projects/2fabcd4bf2fc4e2f8ed4f28302348d24/users/4e919a58ca13432a90fe8367a4b7493d/roles/4559b9a0837249f895a13c70096f8d38 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:33:58.874978 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace337d6-c090-4a3b-a988-4fea338d7347 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:58.901819 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:33:58.938276 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b549f211-1392-490c-b3b3-20f5d396c33f admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:58.951038 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace337d6-c090-4a3b-a988-4fea338d7347 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:58.956721 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed66194-bf88-4108-952f-af2b71ee3ede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:58.957378 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed66194-bf88-4108-952f-af2b71ee3ede admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:58.957662 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed66194-bf88-4108-952f-af2b71ee3ede admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:58.957939 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed66194-bf88-4108-952f-af2b71ee3ede admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:58.959042 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed66194-bf88-4108-952f-af2b71ee3ede admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:58.960267 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed66194-bf88-4108-952f-af2b71ee3ede admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:58.970438 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1006/2006] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:33:58 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:33:58.974916 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed66194-bf88-4108-952f-af2b71ee3ede admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:58.989774 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed66194-bf88-4108-952f-af2b71ee3ede admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:58.997278 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb101a5-f510-4a17-8518-9752093fd705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:58.997951 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb101a5-f510-4a17-8518-9752093fd705 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:58.998236 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb101a5-f510-4a17-8518-9752093fd705 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:58.998517 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb101a5-f510-4a17-8518-9752093fd705 admin admin] PATH_INFO: `/v3/projects/2fabcd4bf2fc4e2f8ed4f28302348d24/users/4e919a58ca13432a90fe8367a4b7493d/roles/4559b9a0837249f895a13c70096f8d38` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:59.000966 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb101a5-f510-4a17-8518-9752093fd705 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fabcd4bf2fc4e2f8ed4f28302348d24, user_id=4e919a58ca13432a90fe8367a4b7493d, role_id=4559b9a0837249f895a13c70096f8d38)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:59.004894 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb101a5-f510-4a17-8518-9752093fd705 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:59.006131 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020bd146-7fe8-46d8-bea8-68129ac63567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:59.006820 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020bd146-7fe8-46d8-bea8-68129ac63567 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:59.007109 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020bd146-7fe8-46d8-bea8-68129ac63567 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:59.007390 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020bd146-7fe8-46d8-bea8-68129ac63567 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:59.008129 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020bd146-7fe8-46d8-bea8-68129ac63567 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:59.010214 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020bd146-7fe8-46d8-bea8-68129ac63567 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:59.016628 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1007/2007] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:33:58 2022] PUT /identity/v3/projects/2fabcd4bf2fc4e2f8ed4f28302348d24/users/4e919a58ca13432a90fe8367a4b7493d/roles/4559b9a0837249f895a13c70096f8d38 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:33:59.022221 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1001/2008] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:33:58 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:33:59.026396 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb101a5-f510-4a17-8518-9752093fd705 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:59.030317 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086bb544-f7f0-447c-a253-95414b15e339 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:59.030607 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086bb544-f7f0-447c-a253-95414b15e339 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:59.030888 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086bb544-f7f0-447c-a253-95414b15e339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:59.098438 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-086bb544-f7f0-447c-a253-95414b15e339 None None] MFA Rules not processed for user `4e919a58ca13432a90fe8367a4b7493d`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:33:59.099926 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-086bb544-f7f0-447c-a253-95414b15e339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:59.133792 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:33:59.168253 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb101a5-f510-4a17-8518-9752093fd705 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:59.180436 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd63ebf-a8c3-4c35-85c1-4ac0c4057fe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:59.181944 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd63ebf-a8c3-4c35-85c1-4ac0c4057fe8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:59.183158 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd63ebf-a8c3-4c35-85c1-4ac0c4057fe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:59.183818 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd63ebf-a8c3-4c35-85c1-4ac0c4057fe8 admin admin] PATH_INFO: `/v3/projects/a1b30dc6b8df42459be65cb8c1e4b27a/users/824d848767534057989e1fb0830b3a83/roles/4559b9a0837249f895a13c70096f8d38` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:59.191299 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd63ebf-a8c3-4c35-85c1-4ac0c4057fe8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1b30dc6b8df42459be65cb8c1e4b27a, user_id=824d848767534057989e1fb0830b3a83, role_id=4559b9a0837249f895a13c70096f8d38)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:59.197004 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd63ebf-a8c3-4c35-85c1-4ac0c4057fe8 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:59.227834 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1008/2009] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:33:59 2022] PUT /identity/v3/projects/a1b30dc6b8df42459be65cb8c1e4b27a/users/824d848767534057989e1fb0830b3a83/roles/4559b9a0837249f895a13c70096f8d38 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:33:59.231700 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad7fa7a-6aa5-49d2-8335-3fa8a366c79c None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:59.232090 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad7fa7a-6aa5-49d2-8335-3fa8a366c79c None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:59.232474 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad7fa7a-6aa5-49d2-8335-3fa8a366c79c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:59.269706 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1002/2010] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:33:59 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:33:59.281952 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086bb544-f7f0-447c-a253-95414b15e339 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:59.284778 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-bad7fa7a-6aa5-49d2-8335-3fa8a366c79c None None] MFA Rules not processed for user `824d848767534057989e1fb0830b3a83`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:33:59.285961 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-086bb544-f7f0-447c-a253-95414b15e339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:59.286106 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-bad7fa7a-6aa5-49d2-8335-3fa8a366c79c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:59.353513 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086bb544-f7f0-447c-a253-95414b15e339 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:59.378811 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1009/2011] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:33:59 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:33:59.401966 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad7fa7a-6aa5-49d2-8335-3fa8a366c79c None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:59.403551 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-bad7fa7a-6aa5-49d2-8335-3fa8a366c79c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:59.435125 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3872f183-87eb-49cc-9e77-b62be6701d52 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1be1e6173e374a60b9e1fbf51a1ab571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:59.436717 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3872f183-87eb-49cc-9e77-b62be6701d52 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:59.437760 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3872f183-87eb-49cc-9e77-b62be6701d52 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:59.438371 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3872f183-87eb-49cc-9e77-b62be6701d52 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:59.443616 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:33:59.528652 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad7fa7a-6aa5-49d2-8335-3fa8a366c79c None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:59.532473 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3b09cd-6834-4fec-95bf-3baade1eed8c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1be1e6173e374a60b9e1fbf51a1ab571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:59.534302 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3b09cd-6834-4fec-95bf-3baade1eed8c service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:59.534607 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3b09cd-6834-4fec-95bf-3baade1eed8c service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:59.534871 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3b09cd-6834-4fec-95bf-3baade1eed8c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:59.544753 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3b09cd-6834-4fec-95bf-3baade1eed8c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:59.546273 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3b09cd-6834-4fec-95bf-3baade1eed8c service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:59.558614 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1010/2012] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:33:59 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 01:33:59.605557 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3872f183-87eb-49cc-9e77-b62be6701d52 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:59.607194 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3872f183-87eb-49cc-9e77-b62be6701d52 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:59.620875 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1003/2013] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:33:59 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:33:59.711425 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3b09cd-6834-4fec-95bf-3baade1eed8c service cinder] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:59.719063 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3b09cd-6834-4fec-95bf-3baade1eed8c service cinder] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:59.723393 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2b7157-401a-4902-8d8e-687d5c93945b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1be1e6173e374a60b9e1fbf51a1ab571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:59.723997 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2b7157-401a-4902-8d8e-687d5c93945b service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:59.724278 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2b7157-401a-4902-8d8e-687d5c93945b service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:59.724697 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2b7157-401a-4902-8d8e-687d5c93945b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:59.792862 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2b7157-401a-4902-8d8e-687d5c93945b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:59.794443 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2b7157-401a-4902-8d8e-687d5c93945b service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:59.803862 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1011/2014] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:33:59 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:33:59.996044 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3872f183-87eb-49cc-9e77-b62be6701d52 service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:34:00.034188 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:34:00.149946 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3872f183-87eb-49cc-9e77-b62be6701d52 service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:34:00.170615 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc710a00-247b-4840-b0c1-4bbcf3fc566a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:34:00.171290 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc710a00-247b-4840-b0c1-4bbcf3fc566a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:00.171691 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc710a00-247b-4840-b0c1-4bbcf3fc566a admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:00.171973 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc710a00-247b-4840-b0c1-4bbcf3fc566a admin admin] PATH_INFO: `/v3/users/29cd637c61d7444bbfca52b13342b801` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:00.175019 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc710a00-247b-4840-b0c1-4bbcf3fc566a admin admin] RBAC: Authorizing `identity:delete_user(user_id=29cd637c61d7444bbfca52b13342b801)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:34:00.176375 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc710a00-247b-4840-b0c1-4bbcf3fc566a admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:34:00.335052 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-bc710a00-247b-4840-b0c1-4bbcf3fc566a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29cd637c61d7444bbfca52b13342b801'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:00.335898 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-bc710a00-247b-4840-b0c1-4bbcf3fc566a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29cd637c61d7444bbfca52b13342b801'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:00.336596 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-bc710a00-247b-4840-b0c1-4bbcf3fc566a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29cd637c61d7444bbfca52b13342b801'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:00.351876 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-bc710a00-247b-4840-b0c1-4bbcf3fc566a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29cd637c61d7444bbfca52b13342b801'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:00.362014 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-bc710a00-247b-4840-b0c1-4bbcf3fc566a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29cd637c61d7444bbfca52b13342b801'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:00.378536 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1004/2015] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Aug 6 01:33:59 2022] DELETE /identity/v3/users/29cd637c61d7444bbfca52b13342b801 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:34:00.383885 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77315e6-86f5-4063-a4d7-7a6ce753c705 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:00.384372 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77315e6-86f5-4063-a4d7-7a6ce753c705 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:00.384630 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77315e6-86f5-4063-a4d7-7a6ce753c705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:00.411314 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-e77315e6-86f5-4063-a4d7-7a6ce753c705 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:34:00.412654 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-e77315e6-86f5-4063-a4d7-7a6ce753c705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:34:00.482411 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1012/2016] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:34:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:34:00.493746 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc710a00-247b-4840-b0c1-4bbcf3fc566a admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:34:00.495194 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-bc710a00-247b-4840-b0c1-4bbcf3fc566a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:34:00.575865 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc710a00-247b-4840-b0c1-4bbcf3fc566a admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:34:00.581226 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33889d8f-d38c-4446-b0b4-1ed9e4f117c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:34:00.582256 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33889d8f-d38c-4446-b0b4-1ed9e4f117c2 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:00.583031 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33889d8f-d38c-4446-b0b4-1ed9e4f117c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:00.583429 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33889d8f-d38c-4446-b0b4-1ed9e4f117c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:00.609195 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33889d8f-d38c-4446-b0b4-1ed9e4f117c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:34:00.611471 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33889d8f-d38c-4446-b0b4-1ed9e4f117c2 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:34:00.626106 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1005/2017] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:34:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:34:01.244939 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77315e6-86f5-4063-a4d7-7a6ce753c705 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:34:01.246607 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-e77315e6-86f5-4063-a4d7-7a6ce753c705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:34:01.266318 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77315e6-86f5-4063-a4d7-7a6ce753c705 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:34:01.281291 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1577c93d-9e00-4a33-9ec5-ae502a1bad2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:34:01.281996 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1577c93d-9e00-4a33-9ec5-ae502a1bad2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:01.282442 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1577c93d-9e00-4a33-9ec5-ae502a1bad2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:01.282601 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1577c93d-9e00-4a33-9ec5-ae502a1bad2e admin admin] PATH_INFO: `/v3/projects/044dd2763cda4ffeb3518f0d622425c4` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:01.284361 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1577c93d-9e00-4a33-9ec5-ae502a1bad2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=044dd2763cda4ffeb3518f0d622425c4)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:34:01.286306 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1577c93d-9e00-4a33-9ec5-ae502a1bad2e admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:34:01.357459 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-1577c93d-9e00-4a33-9ec5-ae502a1bad2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '044dd2763cda4ffeb3518f0d622425c4', 'request_id': 'req-1577c93d-9e00-4a33-9ec5-ae502a1bad2e'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:01.366474 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-1577c93d-9e00-4a33-9ec5-ae502a1bad2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '044dd2763cda4ffeb3518f0d622425c4', 'request_id': 'req-1577c93d-9e00-4a33-9ec5-ae502a1bad2e'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:01.368769 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:34:01.374295 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-1577c93d-9e00-4a33-9ec5-ae502a1bad2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '044dd2763cda4ffeb3518f0d622425c4', 'request_id': 'req-1577c93d-9e00-4a33-9ec5-ae502a1bad2e'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:01.388806 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-1577c93d-9e00-4a33-9ec5-ae502a1bad2e admin admin] The token cache is being invalidate because project 044dd2763cda4ffeb3518f0d622425c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:34:01.389467 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-1577c93d-9e00-4a33-9ec5-ae502a1bad2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:01.389810 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-1577c93d-9e00-4a33-9ec5-ae502a1bad2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:01.391653 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1013/2018] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Aug 6 01:34:01 2022] DELETE /identity/v3/projects/044dd2763cda4ffeb3518f0d622425c4 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:34:01.399784 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33889d8f-d38c-4446-b0b4-1ed9e4f117c2 service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:34:01.411512 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33889d8f-d38c-4446-b0b4-1ed9e4f117c2 service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:34:01.448692 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:34:01.486603 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b951d1ef-3f1f-4230-81fa-92687f1f8edc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:34:01.487464 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b951d1ef-3f1f-4230-81fa-92687f1f8edc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:01.487960 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b951d1ef-3f1f-4230-81fa-92687f1f8edc admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:01.488209 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b951d1ef-3f1f-4230-81fa-92687f1f8edc admin admin] PATH_INFO: `/v3/users/824d848767534057989e1fb0830b3a83` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:01.492669 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b951d1ef-3f1f-4230-81fa-92687f1f8edc admin admin] RBAC: Authorizing `identity:delete_user(user_id=824d848767534057989e1fb0830b3a83)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:34:01.494710 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b951d1ef-3f1f-4230-81fa-92687f1f8edc admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:34:01.551259 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-b951d1ef-3f1f-4230-81fa-92687f1f8edc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '824d848767534057989e1fb0830b3a83'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:01.552719 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-b951d1ef-3f1f-4230-81fa-92687f1f8edc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '824d848767534057989e1fb0830b3a83'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:01.552719 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-b951d1ef-3f1f-4230-81fa-92687f1f8edc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '824d848767534057989e1fb0830b3a83'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:01.567063 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-b951d1ef-3f1f-4230-81fa-92687f1f8edc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '824d848767534057989e1fb0830b3a83'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:01.576460 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-b951d1ef-3f1f-4230-81fa-92687f1f8edc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '824d848767534057989e1fb0830b3a83'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:01.601751 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1006/2019] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Aug 6 01:34:01 2022] DELETE /identity/v3/users/824d848767534057989e1fb0830b3a83 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:34:01.899034 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1577c93d-9e00-4a33-9ec5-ae502a1bad2e admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:34:01.900569 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-1577c93d-9e00-4a33-9ec5-ae502a1bad2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:34:01.915373 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1577c93d-9e00-4a33-9ec5-ae502a1bad2e admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:34:01.978498 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23524e2e-2811-4785-8230-785fcd20f028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:34:01.979152 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23524e2e-2811-4785-8230-785fcd20f028 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:01.979438 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23524e2e-2811-4785-8230-785fcd20f028 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:01.979720 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23524e2e-2811-4785-8230-785fcd20f028 admin admin] PATH_INFO: `/v3/projects/a1b30dc6b8df42459be65cb8c1e4b27a` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:01.981133 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23524e2e-2811-4785-8230-785fcd20f028 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1b30dc6b8df42459be65cb8c1e4b27a)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:34:01.983120 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23524e2e-2811-4785-8230-785fcd20f028 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:34:02.056144 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-23524e2e-2811-4785-8230-785fcd20f028 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1b30dc6b8df42459be65cb8c1e4b27a', 'request_id': 'req-23524e2e-2811-4785-8230-785fcd20f028'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:02.063052 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-23524e2e-2811-4785-8230-785fcd20f028 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1b30dc6b8df42459be65cb8c1e4b27a', 'request_id': 'req-23524e2e-2811-4785-8230-785fcd20f028'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:02.068936 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-23524e2e-2811-4785-8230-785fcd20f028 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1b30dc6b8df42459be65cb8c1e4b27a', 'request_id': 'req-23524e2e-2811-4785-8230-785fcd20f028'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:02.083649 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-23524e2e-2811-4785-8230-785fcd20f028 admin admin] The token cache is being invalidate because project a1b30dc6b8df42459be65cb8c1e4b27a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:34:02.083945 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-23524e2e-2811-4785-8230-785fcd20f028 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:02.084681 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-23524e2e-2811-4785-8230-785fcd20f028 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:02.086748 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1014/2020] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Aug 6 01:34:01 2022] DELETE /identity/v3/projects/a1b30dc6b8df42459be65cb8c1e4b27a => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:34:02.091305 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb1b6e1-7ad7-4952-989c-7e317903e763 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:02.091601 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb1b6e1-7ad7-4952-989c-7e317903e763 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:02.091883 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb1b6e1-7ad7-4952-989c-7e317903e763 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:02.113986 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-cbb1b6e1-7ad7-4952-989c-7e317903e763 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:34:02.115998 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-cbb1b6e1-7ad7-4952-989c-7e317903e763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:34:02.178634 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1007/2021] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:34:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:34:02.186556 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f206db-a8a7-4762-a172-2a2055f99692 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:02.186854 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f206db-a8a7-4762-a172-2a2055f99692 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:02.187139 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f206db-a8a7-4762-a172-2a2055f99692 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:02.207347 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-a0f206db-a8a7-4762-a172-2a2055f99692 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:34:02.208644 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-a0f206db-a8a7-4762-a172-2a2055f99692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:34:02.217206 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1015/2022] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:34:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:34:02.221703 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb1b6e1-7ad7-4952-989c-7e317903e763 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:34:02.231691 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb1b6e1-7ad7-4952-989c-7e317903e763 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:34:02.237953 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e960cb45-d259-4549-9677-26c23789b2e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:34:02.238664 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e960cb45-d259-4549-9677-26c23789b2e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:02.239022 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e960cb45-d259-4549-9677-26c23789b2e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:02.239633 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e960cb45-d259-4549-9677-26c23789b2e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:02.240536 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e960cb45-d259-4549-9677-26c23789b2e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:34:02.240949 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e960cb45-d259-4549-9677-26c23789b2e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:34:02.242808 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e960cb45-d259-4549-9677-26c23789b2e9 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:34:02.256087 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1008/2023] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Aug 6 01:34:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:34:02.260443 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f206db-a8a7-4762-a172-2a2055f99692 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:34:02.270422 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f206db-a8a7-4762-a172-2a2055f99692 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:34:02.275373 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185709d8-4e4a-4bfa-929e-5f0906d65084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:34:02.276130 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185709d8-4e4a-4bfa-929e-5f0906d65084 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:02.276481 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185709d8-4e4a-4bfa-929e-5f0906d65084 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:02.276820 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185709d8-4e4a-4bfa-929e-5f0906d65084 admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:02.277600 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185709d8-4e4a-4bfa-929e-5f0906d65084 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:34:02.279558 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185709d8-4e4a-4bfa-929e-5f0906d65084 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:34:02.307429 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1016/2024] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Aug 6 01:34:02 2022] POST /identity/v3/projects => generated 361 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:34:02.311421 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e960cb45-d259-4549-9677-26c23789b2e9 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:34:02.321684 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e960cb45-d259-4549-9677-26c23789b2e9 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:34:02.363330 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:34:02.382169 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ff0d22-93a6-4f24-8adf-d02015999673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:34:02.382809 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ff0d22-93a6-4f24-8adf-d02015999673 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:02.383097 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ff0d22-93a6-4f24-8adf-d02015999673 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:02.383381 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ff0d22-93a6-4f24-8adf-d02015999673 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:02.384122 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ff0d22-93a6-4f24-8adf-d02015999673 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:34:02.386061 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ff0d22-93a6-4f24-8adf-d02015999673 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:34:02.413317 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1009/2025] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:34:02 2022] POST /identity/v3/users => generated 281 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:34:02.415207 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185709d8-4e4a-4bfa-929e-5f0906d65084 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:34:02.426123 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185709d8-4e4a-4bfa-929e-5f0906d65084 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:34:02.430797 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaeb2011-be40-4e90-98d6-056f1e94dc01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:34:02.431415 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeb2011-be40-4e90-98d6-056f1e94dc01 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:02.431700 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeb2011-be40-4e90-98d6-056f1e94dc01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:02.431977 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeb2011-be40-4e90-98d6-056f1e94dc01 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:02.432684 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaeb2011-be40-4e90-98d6-056f1e94dc01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:34:02.434301 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaeb2011-be40-4e90-98d6-056f1e94dc01 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:34:02.445764 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1017/2026] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:34:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:34:02.449932 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ff0d22-93a6-4f24-8adf-d02015999673 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:34:02.460968 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ff0d22-93a6-4f24-8adf-d02015999673 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:34:02.465699 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6b566f-9c38-4715-8506-97e5408c4782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:34:02.466327 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6b566f-9c38-4715-8506-97e5408c4782 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:02.466613 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6b566f-9c38-4715-8506-97e5408c4782 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:02.466892 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6b566f-9c38-4715-8506-97e5408c4782 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:02.467584 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6b566f-9c38-4715-8506-97e5408c4782 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:34:02.469229 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6b566f-9c38-4715-8506-97e5408c4782 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:34:02.480373 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1010/2027] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:34:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:34:02.484417 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaeb2011-be40-4e90-98d6-056f1e94dc01 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:34:02.494858 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaeb2011-be40-4e90-98d6-056f1e94dc01 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:34:02.499709 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b06cf3d-aff7-4483-bb97-64a077897adf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:34:02.500402 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b06cf3d-aff7-4483-bb97-64a077897adf admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:02.500713 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b06cf3d-aff7-4483-bb97-64a077897adf admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:02.501023 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b06cf3d-aff7-4483-bb97-64a077897adf admin admin] PATH_INFO: `/v3/projects/3408939be8e747ce97cea044bbd8b7e1/users/58321e5dec2b433bb47748cb4a23f95a/roles/61b3b93d37a047e4bb8369b46fe20db9` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:02.516781 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b06cf3d-aff7-4483-bb97-64a077897adf admin admin] RBAC: Authorizing `identity:create_grant(project_id=3408939be8e747ce97cea044bbd8b7e1, user_id=58321e5dec2b433bb47748cb4a23f95a, role_id=61b3b93d37a047e4bb8369b46fe20db9)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:34:02.520393 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b06cf3d-aff7-4483-bb97-64a077897adf admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:34:02.536500 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1018/2028] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:34:02 2022] PUT /identity/v3/projects/3408939be8e747ce97cea044bbd8b7e1/users/58321e5dec2b433bb47748cb4a23f95a/roles/61b3b93d37a047e4bb8369b46fe20db9 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:34:02.547150 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7c2c41-142d-4a91-9700-dec8e78c0fc4 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:02.547511 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7c2c41-142d-4a91-9700-dec8e78c0fc4 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:02.547797 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7c2c41-142d-4a91-9700-dec8e78c0fc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:02.577989 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-8c7c2c41-142d-4a91-9700-dec8e78c0fc4 None None] MFA Rules not processed for user `58321e5dec2b433bb47748cb4a23f95a`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:34:02.579226 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-8c7c2c41-142d-4a91-9700-dec8e78c0fc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:34:02.650390 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1011/2029] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:34:02 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:34:02.680345 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b06cf3d-aff7-4483-bb97-64a077897adf admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:34:02.682556 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-3b06cf3d-aff7-4483-bb97-64a077897adf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:34:02.702864 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:34:02.787289 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b06cf3d-aff7-4483-bb97-64a077897adf admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:34:02.791414 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc91e84c-5726-492e-b146-11a69390d074 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1be1e6173e374a60b9e1fbf51a1ab571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:34:02.791982 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc91e84c-5726-492e-b146-11a69390d074 service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:02.792335 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc91e84c-5726-492e-b146-11a69390d074 service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:02.792687 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc91e84c-5726-492e-b146-11a69390d074 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:02.802422 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc91e84c-5726-492e-b146-11a69390d074 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:34:02.803869 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc91e84c-5726-492e-b146-11a69390d074 service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:34:02.813289 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1019/2030] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:34:02 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:34:02.829131 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7c2c41-142d-4a91-9700-dec8e78c0fc4 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:34:02.903335 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7c2c41-142d-4a91-9700-dec8e78c0fc4 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:34:02.908272 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c0d702-c230-4ec3-b2ef-5ec4fa488c39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:34:02.908989 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c0d702-c230-4ec3-b2ef-5ec4fa488c39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:02.909367 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c0d702-c230-4ec3-b2ef-5ec4fa488c39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:02.909714 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c0d702-c230-4ec3-b2ef-5ec4fa488c39 admin admin] PATH_INFO: `/v3/users/58321e5dec2b433bb47748cb4a23f95a` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:02.910968 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c0d702-c230-4ec3-b2ef-5ec4fa488c39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=58321e5dec2b433bb47748cb4a23f95a)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:34:02.912960 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c0d702-c230-4ec3-b2ef-5ec4fa488c39 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:34:02.966502 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-32c0d702-c230-4ec3-b2ef-5ec4fa488c39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58321e5dec2b433bb47748cb4a23f95a'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:02.967211 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-32c0d702-c230-4ec3-b2ef-5ec4fa488c39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58321e5dec2b433bb47748cb4a23f95a'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:02.967786 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-32c0d702-c230-4ec3-b2ef-5ec4fa488c39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58321e5dec2b433bb47748cb4a23f95a'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:02.976683 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:34:02.988886 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-32c0d702-c230-4ec3-b2ef-5ec4fa488c39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58321e5dec2b433bb47748cb4a23f95a'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:02.995987 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-32c0d702-c230-4ec3-b2ef-5ec4fa488c39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58321e5dec2b433bb47748cb4a23f95a'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:03.010954 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1012/2031] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Aug 6 01:34:02 2022] DELETE /identity/v3/users/58321e5dec2b433bb47748cb4a23f95a => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:34:03.015313 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f37fd12-3ba3-4f90-9bdf-6fa540262c34 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:03.015606 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f37fd12-3ba3-4f90-9bdf-6fa540262c34 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:03.015884 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f37fd12-3ba3-4f90-9bdf-6fa540262c34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:03.037158 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-1f37fd12-3ba3-4f90-9bdf-6fa540262c34 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:34:03.037158 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-1f37fd12-3ba3-4f90-9bdf-6fa540262c34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:34:03.116722 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1020/2032] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:34:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:34:03.118526 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c0d702-c230-4ec3-b2ef-5ec4fa488c39 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:34:03.122685 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-32c0d702-c230-4ec3-b2ef-5ec4fa488c39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:34:03.198463 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c0d702-c230-4ec3-b2ef-5ec4fa488c39 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:34:03.202183 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa265f23-eeab-48cc-b564-794190613678 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:34:03.202810 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa265f23-eeab-48cc-b564-794190613678 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:03.203096 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa265f23-eeab-48cc-b564-794190613678 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:03.203377 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa265f23-eeab-48cc-b564-794190613678 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:03.222422 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa265f23-eeab-48cc-b564-794190613678 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:34:03.223877 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa265f23-eeab-48cc-b564-794190613678 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:34:03.235139 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1013/2033] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:34:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:34:03.932974 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f37fd12-3ba3-4f90-9bdf-6fa540262c34 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:34:03.937183 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-1f37fd12-3ba3-4f90-9bdf-6fa540262c34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:34:03.964667 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f37fd12-3ba3-4f90-9bdf-6fa540262c34 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:34:03.972404 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c34f045-910e-4443-a436-fa9a4bd67aac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:34:03.973623 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c34f045-910e-4443-a436-fa9a4bd67aac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:03.974122 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c34f045-910e-4443-a436-fa9a4bd67aac admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:03.974410 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c34f045-910e-4443-a436-fa9a4bd67aac admin admin] PATH_INFO: `/v3/projects/3408939be8e747ce97cea044bbd8b7e1` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:03.975613 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c34f045-910e-4443-a436-fa9a4bd67aac admin admin] RBAC: Authorizing `identity:delete_project(project_id=3408939be8e747ce97cea044bbd8b7e1)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:34:03.978056 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c34f045-910e-4443-a436-fa9a4bd67aac admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:34:04.032345 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-8c34f045-910e-4443-a436-fa9a4bd67aac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3408939be8e747ce97cea044bbd8b7e1', 'request_id': 'req-8c34f045-910e-4443-a436-fa9a4bd67aac'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:04.039615 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-8c34f045-910e-4443-a436-fa9a4bd67aac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3408939be8e747ce97cea044bbd8b7e1', 'request_id': 'req-8c34f045-910e-4443-a436-fa9a4bd67aac'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:04.046861 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-8c34f045-910e-4443-a436-fa9a4bd67aac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3408939be8e747ce97cea044bbd8b7e1', 'request_id': 'req-8c34f045-910e-4443-a436-fa9a4bd67aac'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:04.062191 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-8c34f045-910e-4443-a436-fa9a4bd67aac admin admin] The token cache is being invalidate because project 3408939be8e747ce97cea044bbd8b7e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:34:04.062616 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-8c34f045-910e-4443-a436-fa9a4bd67aac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:04.063262 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-8c34f045-910e-4443-a436-fa9a4bd67aac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:04.065287 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1021/2034] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Aug 6 01:34:03 2022] DELETE /identity/v3/projects/3408939be8e747ce97cea044bbd8b7e1 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:34:04.070618 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b19940-cd8b-4413-b9d0-13772f7c772b None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:04.071108 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b19940-cd8b-4413-b9d0-13772f7c772b None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:04.071331 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b19940-cd8b-4413-b9d0-13772f7c772b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:04.092091 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-95b19940-cd8b-4413-b9d0-13772f7c772b None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:34:04.093460 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-95b19940-cd8b-4413-b9d0-13772f7c772b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:34:04.199120 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1014/2035] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:34:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:34:04.208060 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27456f1b-44e5-46f5-b681-a7c7d2466ba6 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:04.208359 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27456f1b-44e5-46f5-b681-a7c7d2466ba6 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:04.208719 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27456f1b-44e5-46f5-b681-a7c7d2466ba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:04.230585 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-27456f1b-44e5-46f5-b681-a7c7d2466ba6 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:34:04.232121 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-27456f1b-44e5-46f5-b681-a7c7d2466ba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:34:04.241664 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1022/2036] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:34:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:34:04.246465 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b19940-cd8b-4413-b9d0-13772f7c772b None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:34:04.262732 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b19940-cd8b-4413-b9d0-13772f7c772b None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:34:04.269279 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686b2ab0-6013-4e80-b0eb-dd485499d62b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:34:04.269915 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686b2ab0-6013-4e80-b0eb-dd485499d62b admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:04.270226 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686b2ab0-6013-4e80-b0eb-dd485499d62b admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:04.270506 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686b2ab0-6013-4e80-b0eb-dd485499d62b admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:04.271222 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686b2ab0-6013-4e80-b0eb-dd485499d62b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:34:04.271529 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686b2ab0-6013-4e80-b0eb-dd485499d62b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:34:04.273768 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686b2ab0-6013-4e80-b0eb-dd485499d62b admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:34:04.287588 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1015/2037] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Aug 6 01:34:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:34:04.291990 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27456f1b-44e5-46f5-b681-a7c7d2466ba6 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:34:04.301569 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27456f1b-44e5-46f5-b681-a7c7d2466ba6 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:34:04.306208 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ac2dd8-ec76-418a-b0ee-70bb9be83f46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:34:04.306823 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ac2dd8-ec76-418a-b0ee-70bb9be83f46 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:04.307111 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ac2dd8-ec76-418a-b0ee-70bb9be83f46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:04.308042 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ac2dd8-ec76-418a-b0ee-70bb9be83f46 admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:04.308172 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ac2dd8-ec76-418a-b0ee-70bb9be83f46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:34:04.310121 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ac2dd8-ec76-418a-b0ee-70bb9be83f46 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:34:04.339076 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1023/2038] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Aug 6 01:34:04 2022] POST /identity/v3/projects => generated 389 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:34:04.344302 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686b2ab0-6013-4e80-b0eb-dd485499d62b admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:34:04.356614 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686b2ab0-6013-4e80-b0eb-dd485499d62b admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:34:04.416482 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4ffdd4-336b-4995-9b1d-d429c1e54b90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:34:04.417083 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4ffdd4-336b-4995-9b1d-d429c1e54b90 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:04.417370 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4ffdd4-336b-4995-9b1d-d429c1e54b90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:04.417644 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4ffdd4-336b-4995-9b1d-d429c1e54b90 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:04.418305 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4ffdd4-336b-4995-9b1d-d429c1e54b90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:34:04.420245 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4ffdd4-336b-4995-9b1d-d429c1e54b90 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:34:04.428939 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:34:04.442235 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1016/2039] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:34:04 2022] POST /identity/v3/users => generated 295 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:34:04.446362 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ac2dd8-ec76-418a-b0ee-70bb9be83f46 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:34:04.457219 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ac2dd8-ec76-418a-b0ee-70bb9be83f46 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:34:04.461819 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23746063-2a00-4863-81d4-849916d56efb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:34:04.462433 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23746063-2a00-4863-81d4-849916d56efb admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:04.462717 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23746063-2a00-4863-81d4-849916d56efb admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:04.462986 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23746063-2a00-4863-81d4-849916d56efb admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:04.463841 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23746063-2a00-4863-81d4-849916d56efb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:34:04.465445 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23746063-2a00-4863-81d4-849916d56efb admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:34:04.467476 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:34:04.480786 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1024/2040] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:34:04 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:34:04.481862 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4ffdd4-336b-4995-9b1d-d429c1e54b90 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:34:04.492187 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4ffdd4-336b-4995-9b1d-d429c1e54b90 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:34:04.496834 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0a639f-0530-4de6-9eeb-53c51f335304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:34:04.497481 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0a639f-0530-4de6-9eeb-53c51f335304 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:04.497766 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0a639f-0530-4de6-9eeb-53c51f335304 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:04.498044 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0a639f-0530-4de6-9eeb-53c51f335304 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:04.498736 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0a639f-0530-4de6-9eeb-53c51f335304 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:34:04.500328 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0a639f-0530-4de6-9eeb-53c51f335304 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:34:04.510894 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1017/2041] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:34:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:34:04.514898 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23746063-2a00-4863-81d4-849916d56efb admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:34:04.525403 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23746063-2a00-4863-81d4-849916d56efb admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:34:04.530026 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedd8121-ff65-48cb-ba95-216ed506bd0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:34:04.530679 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedd8121-ff65-48cb-ba95-216ed506bd0e admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:04.530963 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedd8121-ff65-48cb-ba95-216ed506bd0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:04.531236 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedd8121-ff65-48cb-ba95-216ed506bd0e admin admin] PATH_INFO: `/v3/projects/d681586de5344700900aa8d0cd9d9b6d/users/8f51517b1e95427d9c6649582c253d41/roles/61b3b93d37a047e4bb8369b46fe20db9` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:04.546207 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedd8121-ff65-48cb-ba95-216ed506bd0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d681586de5344700900aa8d0cd9d9b6d, user_id=8f51517b1e95427d9c6649582c253d41, role_id=61b3b93d37a047e4bb8369b46fe20db9)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:34:04.549186 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedd8121-ff65-48cb-ba95-216ed506bd0e admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:34:04.562599 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1025/2042] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:34:04 2022] PUT /identity/v3/projects/d681586de5344700900aa8d0cd9d9b6d/users/8f51517b1e95427d9c6649582c253d41/roles/61b3b93d37a047e4bb8369b46fe20db9 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:34:04.573333 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16f9d31-6ae9-48ae-b119-1cceb296f39a None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:04.573652 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16f9d31-6ae9-48ae-b119-1cceb296f39a None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:04.573933 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16f9d31-6ae9-48ae-b119-1cceb296f39a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:04.602790 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-d16f9d31-6ae9-48ae-b119-1cceb296f39a None None] MFA Rules not processed for user `8f51517b1e95427d9c6649582c253d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:34:04.604006 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-d16f9d31-6ae9-48ae-b119-1cceb296f39a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:34:04.668194 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1018/2043] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:34:04 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:34:04.679560 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedd8121-ff65-48cb-ba95-216ed506bd0e admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:34:04.681603 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-bedd8121-ff65-48cb-ba95-216ed506bd0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:34:04.746183 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedd8121-ff65-48cb-ba95-216ed506bd0e admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:34:04.749860 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36767078-c523-454d-862d-1c264367ec05 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1be1e6173e374a60b9e1fbf51a1ab571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:34:04.750484 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36767078-c523-454d-862d-1c264367ec05 service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:04.750779 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36767078-c523-454d-862d-1c264367ec05 service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:04.751105 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36767078-c523-454d-862d-1c264367ec05 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:04.760729 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36767078-c523-454d-862d-1c264367ec05 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:34:04.763395 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36767078-c523-454d-862d-1c264367ec05 service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:34:04.776156 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1026/2044] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:34:04 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:34:05.084639 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16f9d31-6ae9-48ae-b119-1cceb296f39a None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:34:05.086559 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-d16f9d31-6ae9-48ae-b119-1cceb296f39a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:34:05.213455 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16f9d31-6ae9-48ae-b119-1cceb296f39a None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:34:05.218078 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e39dc3e-2882-4279-9f49-fa162b7806df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:34:05.218720 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e39dc3e-2882-4279-9f49-fa162b7806df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:05.219001 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e39dc3e-2882-4279-9f49-fa162b7806df admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:05.219276 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e39dc3e-2882-4279-9f49-fa162b7806df admin admin] PATH_INFO: `/v3/users/d77a4cd10b264ddeac77994f0d690f29` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:05.221134 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e39dc3e-2882-4279-9f49-fa162b7806df admin admin] RBAC: Authorizing `identity:delete_user(user_id=d77a4cd10b264ddeac77994f0d690f29)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:34:05.223218 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e39dc3e-2882-4279-9f49-fa162b7806df admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:34:05.281695 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-4e39dc3e-2882-4279-9f49-fa162b7806df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd77a4cd10b264ddeac77994f0d690f29'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:05.282459 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-4e39dc3e-2882-4279-9f49-fa162b7806df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd77a4cd10b264ddeac77994f0d690f29'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:05.283722 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-4e39dc3e-2882-4279-9f49-fa162b7806df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd77a4cd10b264ddeac77994f0d690f29'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:05.296065 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-4e39dc3e-2882-4279-9f49-fa162b7806df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd77a4cd10b264ddeac77994f0d690f29'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:05.302260 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:INSERT=2,keystone:SELECT=89,keystone:DELETE=9 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:34:05.304358 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-4e39dc3e-2882-4279-9f49-fa162b7806df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd77a4cd10b264ddeac77994f0d690f29'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:05.323612 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1019/2045] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Aug 6 01:34:05 2022] DELETE /identity/v3/users/d77a4cd10b264ddeac77994f0d690f29 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:34:05.325723 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffa275b-9d21-450a-a950-4e0e8dd78f71 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:05.326550 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffa275b-9d21-450a-a950-4e0e8dd78f71 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:05.327138 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffa275b-9d21-450a-a950-4e0e8dd78f71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:05.351381 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-5ffa275b-9d21-450a-a950-4e0e8dd78f71 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:34:05.352604 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-5ffa275b-9d21-450a-a950-4e0e8dd78f71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:34:05.442957 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1027/2046] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:34:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:34:05.458298 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e39dc3e-2882-4279-9f49-fa162b7806df admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:34:05.459924 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-4e39dc3e-2882-4279-9f49-fa162b7806df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:34:05.523373 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e39dc3e-2882-4279-9f49-fa162b7806df admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:34:05.527067 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3505de-0125-4985-b4a0-18e2be2c11f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:34:05.527677 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3505de-0125-4985-b4a0-18e2be2c11f9 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:05.527959 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3505de-0125-4985-b4a0-18e2be2c11f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:05.528274 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3505de-0125-4985-b4a0-18e2be2c11f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:05.537879 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3505de-0125-4985-b4a0-18e2be2c11f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:34:05.539553 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3505de-0125-4985-b4a0-18e2be2c11f9 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:34:05.549476 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1020/2047] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:34:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:34:05.866746 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffa275b-9d21-450a-a950-4e0e8dd78f71 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:34:05.868232 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-5ffa275b-9d21-450a-a950-4e0e8dd78f71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:34:05.881565 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffa275b-9d21-450a-a950-4e0e8dd78f71 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:34:05.886151 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37602dae-165a-4edb-b7d0-880780096ebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:34:05.887073 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37602dae-165a-4edb-b7d0-880780096ebd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:05.887352 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37602dae-165a-4edb-b7d0-880780096ebd admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:05.887623 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37602dae-165a-4edb-b7d0-880780096ebd admin admin] PATH_INFO: `/v3/projects/34d1896282a044b4a37fcd56cf14b14e` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:05.888783 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37602dae-165a-4edb-b7d0-880780096ebd admin admin] RBAC: Authorizing `identity:delete_project(project_id=34d1896282a044b4a37fcd56cf14b14e)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:34:05.890748 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37602dae-165a-4edb-b7d0-880780096ebd admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:34:05.937677 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-37602dae-165a-4edb-b7d0-880780096ebd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34d1896282a044b4a37fcd56cf14b14e', 'request_id': 'req-37602dae-165a-4edb-b7d0-880780096ebd'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:05.944343 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-37602dae-165a-4edb-b7d0-880780096ebd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34d1896282a044b4a37fcd56cf14b14e', 'request_id': 'req-37602dae-165a-4edb-b7d0-880780096ebd'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:05.949895 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-37602dae-165a-4edb-b7d0-880780096ebd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34d1896282a044b4a37fcd56cf14b14e', 'request_id': 'req-37602dae-165a-4edb-b7d0-880780096ebd'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:05.958479 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:34:05.965559 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-37602dae-165a-4edb-b7d0-880780096ebd admin admin] The token cache is being invalidate because project 34d1896282a044b4a37fcd56cf14b14e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:34:05.966059 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-37602dae-165a-4edb-b7d0-880780096ebd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:05.967371 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-37602dae-165a-4edb-b7d0-880780096ebd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:05.969501 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1028/2048] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Aug 6 01:34:05 2022] DELETE /identity/v3/projects/34d1896282a044b4a37fcd56cf14b14e => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:34:05.973509 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3505de-0125-4985-b4a0-18e2be2c11f9 service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:34:05.983445 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3505de-0125-4985-b4a0-18e2be2c11f9 service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:34:06.046541 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7333df-fcc6-4f67-8b68-809bbae834f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:34:06.047200 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7333df-fcc6-4f67-8b68-809bbae834f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:06.047487 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7333df-fcc6-4f67-8b68-809bbae834f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:06.047767 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7333df-fcc6-4f67-8b68-809bbae834f9 admin admin] PATH_INFO: `/v3/users/4e919a58ca13432a90fe8367a4b7493d` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:06.049629 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7333df-fcc6-4f67-8b68-809bbae834f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e919a58ca13432a90fe8367a4b7493d)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:34:06.051584 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7333df-fcc6-4f67-8b68-809bbae834f9 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:34:06.105640 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-3d7333df-fcc6-4f67-8b68-809bbae834f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e919a58ca13432a90fe8367a4b7493d'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:06.106657 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-3d7333df-fcc6-4f67-8b68-809bbae834f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e919a58ca13432a90fe8367a4b7493d'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:06.107383 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-3d7333df-fcc6-4f67-8b68-809bbae834f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e919a58ca13432a90fe8367a4b7493d'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:06.120901 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-3d7333df-fcc6-4f67-8b68-809bbae834f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e919a58ca13432a90fe8367a4b7493d'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:06.127881 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-3d7333df-fcc6-4f67-8b68-809bbae834f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e919a58ca13432a90fe8367a4b7493d'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:06.146086 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1021/2049] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Aug 6 01:34:05 2022] DELETE /identity/v3/users/4e919a58ca13432a90fe8367a4b7493d => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:34:06.522394 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37602dae-165a-4edb-b7d0-880780096ebd admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:34:06.524101 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-37602dae-165a-4edb-b7d0-880780096ebd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:34:06.538495 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37602dae-165a-4edb-b7d0-880780096ebd admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:34:06.596366 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4438c080-309c-4c1c-911d-63ec1956d630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:34:06.597007 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4438c080-309c-4c1c-911d-63ec1956d630 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:06.597327 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4438c080-309c-4c1c-911d-63ec1956d630 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:06.597612 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4438c080-309c-4c1c-911d-63ec1956d630 admin admin] PATH_INFO: `/v3/projects/2fabcd4bf2fc4e2f8ed4f28302348d24` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:06.598808 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4438c080-309c-4c1c-911d-63ec1956d630 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2fabcd4bf2fc4e2f8ed4f28302348d24)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:34:06.600761 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4438c080-309c-4c1c-911d-63ec1956d630 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:34:06.648326 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-4438c080-309c-4c1c-911d-63ec1956d630 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2fabcd4bf2fc4e2f8ed4f28302348d24', 'request_id': 'req-4438c080-309c-4c1c-911d-63ec1956d630'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:06.654428 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-4438c080-309c-4c1c-911d-63ec1956d630 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2fabcd4bf2fc4e2f8ed4f28302348d24', 'request_id': 'req-4438c080-309c-4c1c-911d-63ec1956d630'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:06.660109 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-4438c080-309c-4c1c-911d-63ec1956d630 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2fabcd4bf2fc4e2f8ed4f28302348d24', 'request_id': 'req-4438c080-309c-4c1c-911d-63ec1956d630'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:06.672071 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-4438c080-309c-4c1c-911d-63ec1956d630 admin admin] The token cache is being invalidate because project 2fabcd4bf2fc4e2f8ed4f28302348d24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:34:06.672362 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-4438c080-309c-4c1c-911d-63ec1956d630 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:06.673025 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-4438c080-309c-4c1c-911d-63ec1956d630 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:06.676174 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1029/2050] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Aug 6 01:34:06 2022] DELETE /identity/v3/projects/2fabcd4bf2fc4e2f8ed4f28302348d24 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:34:06.679932 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b8f11f-fbfa-4874-aa8e-2d3eca653d4e None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:06.680226 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b8f11f-fbfa-4874-aa8e-2d3eca653d4e None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:06.680504 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b8f11f-fbfa-4874-aa8e-2d3eca653d4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:06.701935 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-83b8f11f-fbfa-4874-aa8e-2d3eca653d4e None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:34:06.703141 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-83b8f11f-fbfa-4874-aa8e-2d3eca653d4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:34:06.709030 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:34:06.766744 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1022/2051] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:34:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:34:06.774712 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41ea524-17c4-4530-abe8-4bb6559bafd3 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:06.775029 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41ea524-17c4-4530-abe8-4bb6559bafd3 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:06.775334 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41ea524-17c4-4530-abe8-4bb6559bafd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:06.795364 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-c41ea524-17c4-4530-abe8-4bb6559bafd3 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:34:06.796589 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-c41ea524-17c4-4530-abe8-4bb6559bafd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:34:06.805701 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1030/2052] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:34:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:34:06.809925 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b8f11f-fbfa-4874-aa8e-2d3eca653d4e None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:34:06.820435 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b8f11f-fbfa-4874-aa8e-2d3eca653d4e None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:34:06.825163 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0ccee7-9a08-4a05-ab70-0adedd23c8f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:34:06.825941 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0ccee7-9a08-4a05-ab70-0adedd23c8f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:06.826057 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0ccee7-9a08-4a05-ab70-0adedd23c8f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:06.826324 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0ccee7-9a08-4a05-ab70-0adedd23c8f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:06.827224 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0ccee7-9a08-4a05-ab70-0adedd23c8f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:34:06.827532 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0ccee7-9a08-4a05-ab70-0adedd23c8f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:34:06.829146 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0ccee7-9a08-4a05-ab70-0adedd23c8f5 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:34:06.842618 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1023/2053] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Aug 6 01:34:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:34:06.847404 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41ea524-17c4-4530-abe8-4bb6559bafd3 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:34:06.856825 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41ea524-17c4-4530-abe8-4bb6559bafd3 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:34:06.861630 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1478ccb-9be8-4cc4-9ba8-04fbb52c4912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:34:06.862246 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1478ccb-9be8-4cc4-9ba8-04fbb52c4912 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:06.862529 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1478ccb-9be8-4cc4-9ba8-04fbb52c4912 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:06.862809 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1478ccb-9be8-4cc4-9ba8-04fbb52c4912 admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:06.863514 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1478ccb-9be8-4cc4-9ba8-04fbb52c4912 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:34:06.865800 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1478ccb-9be8-4cc4-9ba8-04fbb52c4912 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:34:06.896015 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1031/2054] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Aug 6 01:34:06 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:34:06.899904 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0ccee7-9a08-4a05-ab70-0adedd23c8f5 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:34:06.910222 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0ccee7-9a08-4a05-ab70-0adedd23c8f5 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:34:06.965224 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1aea4a-4f70-4fd7-b6d5-c88fb2eb3d0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:34:06.965854 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1aea4a-4f70-4fd7-b6d5-c88fb2eb3d0d admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:06.966141 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1aea4a-4f70-4fd7-b6d5-c88fb2eb3d0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:06.966421 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1aea4a-4f70-4fd7-b6d5-c88fb2eb3d0d admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:06.967085 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1aea4a-4f70-4fd7-b6d5-c88fb2eb3d0d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:34:06.968999 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1aea4a-4f70-4fd7-b6d5-c88fb2eb3d0d admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:34:06.989570 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1024/2055] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:34:06 2022] POST /identity/v3/users => generated 293 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:34:06.993642 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1478ccb-9be8-4cc4-9ba8-04fbb52c4912 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:34:07.004688 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1478ccb-9be8-4cc4-9ba8-04fbb52c4912 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:34:07.009767 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce4d08d-ad6e-4472-ba9d-fbdd847c1b6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:34:07.010323 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce4d08d-ad6e-4472-ba9d-fbdd847c1b6a admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:07.010607 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce4d08d-ad6e-4472-ba9d-fbdd847c1b6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:07.010911 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce4d08d-ad6e-4472-ba9d-fbdd847c1b6a admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:07.011612 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce4d08d-ad6e-4472-ba9d-fbdd847c1b6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:34:07.013240 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce4d08d-ad6e-4472-ba9d-fbdd847c1b6a admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:34:07.023747 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1032/2056] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:34:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:34:07.027593 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1aea4a-4f70-4fd7-b6d5-c88fb2eb3d0d admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:34:07.037816 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1aea4a-4f70-4fd7-b6d5-c88fb2eb3d0d admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:34:07.042507 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3f87d3-5f19-4786-ae36-b0ef1193f28a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:34:07.043105 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3f87d3-5f19-4786-ae36-b0ef1193f28a admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:07.043386 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3f87d3-5f19-4786-ae36-b0ef1193f28a admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:07.043662 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3f87d3-5f19-4786-ae36-b0ef1193f28a admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:07.044344 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3f87d3-5f19-4786-ae36-b0ef1193f28a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:34:07.045961 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3f87d3-5f19-4786-ae36-b0ef1193f28a admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:34:07.056835 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1025/2057] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:34:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:34:07.060893 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce4d08d-ad6e-4472-ba9d-fbdd847c1b6a admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:34:07.072328 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce4d08d-ad6e-4472-ba9d-fbdd847c1b6a admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:34:07.077173 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89b0630-dd55-4f37-a274-f60d0628bdb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:34:07.077830 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89b0630-dd55-4f37-a274-f60d0628bdb0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:07.078113 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89b0630-dd55-4f37-a274-f60d0628bdb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:07.078448 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89b0630-dd55-4f37-a274-f60d0628bdb0 admin admin] PATH_INFO: `/v3/projects/2ee0b748a9b6423a953717e8936809aa/users/16e18403f6ff4b87a9400c363aea892d/roles/61b3b93d37a047e4bb8369b46fe20db9` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:07.093466 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89b0630-dd55-4f37-a274-f60d0628bdb0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ee0b748a9b6423a953717e8936809aa, user_id=16e18403f6ff4b87a9400c363aea892d, role_id=61b3b93d37a047e4bb8369b46fe20db9)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:34:07.096266 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89b0630-dd55-4f37-a274-f60d0628bdb0 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:34:07.109250 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1033/2058] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:34:07 2022] PUT /identity/v3/projects/2ee0b748a9b6423a953717e8936809aa/users/16e18403f6ff4b87a9400c363aea892d/roles/61b3b93d37a047e4bb8369b46fe20db9 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:34:07.120946 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba15994-04ce-491d-8748-b0d4bac77c75 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:07.121441 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba15994-04ce-491d-8748-b0d4bac77c75 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:07.121543 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba15994-04ce-491d-8748-b0d4bac77c75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:07.151005 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-dba15994-04ce-491d-8748-b0d4bac77c75 None None] MFA Rules not processed for user `16e18403f6ff4b87a9400c363aea892d`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:34:07.152231 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-dba15994-04ce-491d-8748-b0d4bac77c75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:34:07.214971 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1026/2059] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:34:07 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:34:07.226466 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89b0630-dd55-4f37-a274-f60d0628bdb0 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:34:07.228215 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-d89b0630-dd55-4f37-a274-f60d0628bdb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:34:07.260986 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:34:07.304842 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89b0630-dd55-4f37-a274-f60d0628bdb0 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:34:07.309000 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc1ae5f-9a4d-406c-bdae-1afa482af841 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1be1e6173e374a60b9e1fbf51a1ab571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:34:07.309869 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc1ae5f-9a4d-406c-bdae-1afa482af841 service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:07.310160 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc1ae5f-9a4d-406c-bdae-1afa482af841 service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:07.310435 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc1ae5f-9a4d-406c-bdae-1afa482af841 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:07.320949 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc1ae5f-9a4d-406c-bdae-1afa482af841 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:34:07.322480 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc1ae5f-9a4d-406c-bdae-1afa482af841 service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:34:07.333935 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1034/2060] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:34:07 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:34:17.211834 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=94,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:34:17.329282 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=14 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:34:30.271951 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba15994-04ce-491d-8748-b0d4bac77c75 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:34:30.333735 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba15994-04ce-491d-8748-b0d4bac77c75 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:34:30.338264 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f29150-5b68-47b2-a1ac-52ca066b281f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:34:30.338907 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f29150-5b68-47b2-a1ac-52ca066b281f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:30.339190 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f29150-5b68-47b2-a1ac-52ca066b281f admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:30.339497 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f29150-5b68-47b2-a1ac-52ca066b281f admin admin] PATH_INFO: `/v3/users/16e18403f6ff4b87a9400c363aea892d` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:30.340674 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f29150-5b68-47b2-a1ac-52ca066b281f admin admin] RBAC: Authorizing `identity:delete_user(user_id=16e18403f6ff4b87a9400c363aea892d)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:34:30.342635 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f29150-5b68-47b2-a1ac-52ca066b281f admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:34:30.392568 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-f3f29150-5b68-47b2-a1ac-52ca066b281f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16e18403f6ff4b87a9400c363aea892d'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:30.393263 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-f3f29150-5b68-47b2-a1ac-52ca066b281f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16e18403f6ff4b87a9400c363aea892d'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:30.393907 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-f3f29150-5b68-47b2-a1ac-52ca066b281f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16e18403f6ff4b87a9400c363aea892d'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:30.406724 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-f3f29150-5b68-47b2-a1ac-52ca066b281f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16e18403f6ff4b87a9400c363aea892d'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:30.415172 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-f3f29150-5b68-47b2-a1ac-52ca066b281f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16e18403f6ff4b87a9400c363aea892d'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:30.428182 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1027/2061] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Aug 6 01:34:30 2022] DELETE /identity/v3/users/16e18403f6ff4b87a9400c363aea892d => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:34:30.432632 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18812875-0a33-43d9-94df-5bab1303d09d None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:30.433196 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18812875-0a33-43d9-94df-5bab1303d09d None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:30.433196 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18812875-0a33-43d9-94df-5bab1303d09d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:30.456551 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-18812875-0a33-43d9-94df-5bab1303d09d None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:34:30.457910 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-18812875-0a33-43d9-94df-5bab1303d09d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:34:30.517428 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1035/2062] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:34:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:34:30.527647 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f29150-5b68-47b2-a1ac-52ca066b281f admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:34:30.529072 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-f3f29150-5b68-47b2-a1ac-52ca066b281f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:34:30.587263 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f29150-5b68-47b2-a1ac-52ca066b281f admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:34:30.591028 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1f0166-02b7-42ac-a3e0-208bbdbc951e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:34:30.591647 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1f0166-02b7-42ac-a3e0-208bbdbc951e service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:30.591935 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1f0166-02b7-42ac-a3e0-208bbdbc951e service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:30.592212 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1f0166-02b7-42ac-a3e0-208bbdbc951e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:30.601441 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1f0166-02b7-42ac-a3e0-208bbdbc951e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:34:30.602876 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1f0166-02b7-42ac-a3e0-208bbdbc951e service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:34:30.612944 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1028/2063] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:34:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:34:31.074379 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18812875-0a33-43d9-94df-5bab1303d09d None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:34:31.075821 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-18812875-0a33-43d9-94df-5bab1303d09d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:34:31.090244 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18812875-0a33-43d9-94df-5bab1303d09d None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:34:31.097427 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6a30a2-bfc0-488f-860e-85a9eda9fda1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:34:31.098430 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6a30a2-bfc0-488f-860e-85a9eda9fda1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:31.098842 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6a30a2-bfc0-488f-860e-85a9eda9fda1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:31.099143 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6a30a2-bfc0-488f-860e-85a9eda9fda1 admin admin] PATH_INFO: `/v3/projects/2ee0b748a9b6423a953717e8936809aa` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:31.101355 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6a30a2-bfc0-488f-860e-85a9eda9fda1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ee0b748a9b6423a953717e8936809aa)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:34:31.103262 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6a30a2-bfc0-488f-860e-85a9eda9fda1 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:34:31.186147 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-8a6a30a2-bfc0-488f-860e-85a9eda9fda1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ee0b748a9b6423a953717e8936809aa', 'request_id': 'req-8a6a30a2-bfc0-488f-860e-85a9eda9fda1'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:31.194450 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-8a6a30a2-bfc0-488f-860e-85a9eda9fda1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ee0b748a9b6423a953717e8936809aa', 'request_id': 'req-8a6a30a2-bfc0-488f-860e-85a9eda9fda1'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:31.201084 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-8a6a30a2-bfc0-488f-860e-85a9eda9fda1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ee0b748a9b6423a953717e8936809aa', 'request_id': 'req-8a6a30a2-bfc0-488f-860e-85a9eda9fda1'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:31.216460 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-8a6a30a2-bfc0-488f-860e-85a9eda9fda1 admin admin] The token cache is being invalidate because project 2ee0b748a9b6423a953717e8936809aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:34:31.217253 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-8a6a30a2-bfc0-488f-860e-85a9eda9fda1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:31.218307 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-8a6a30a2-bfc0-488f-860e-85a9eda9fda1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:31.220326 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1036/2064] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Aug 6 01:34:31 2022] DELETE /identity/v3/projects/2ee0b748a9b6423a953717e8936809aa => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:34:31.225557 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bc340d-9162-4112-a55d-54c2ccc0c760 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:31.225848 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bc340d-9162-4112-a55d-54c2ccc0c760 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:31.226126 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bc340d-9162-4112-a55d-54c2ccc0c760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:31.248401 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-c8bc340d-9162-4112-a55d-54c2ccc0c760 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:34:31.249704 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-c8bc340d-9162-4112-a55d-54c2ccc0c760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:34:31.273979 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:34:31.339211 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1029/2065] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:34:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:34:31.342991 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8d322c-cd3c-48bd-9327-5cafd2c18d28 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:31.343279 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8d322c-cd3c-48bd-9327-5cafd2c18d28 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:31.343555 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8d322c-cd3c-48bd-9327-5cafd2c18d28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:31.367180 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-5d8d322c-cd3c-48bd-9327-5cafd2c18d28 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:34:31.369227 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-5d8d322c-cd3c-48bd-9327-5cafd2c18d28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:34:31.376557 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1037/2066] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:34:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:34:31.380859 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bc340d-9162-4112-a55d-54c2ccc0c760 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:34:31.391787 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bc340d-9162-4112-a55d-54c2ccc0c760 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:34:31.413857 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c069674-b18d-4206-9453-e1a3b7c031de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:34:31.414611 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c069674-b18d-4206-9453-e1a3b7c031de admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:31.415040 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c069674-b18d-4206-9453-e1a3b7c031de admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:31.415222 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c069674-b18d-4206-9453-e1a3b7c031de admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:31.415900 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c069674-b18d-4206-9453-e1a3b7c031de admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:34:31.416239 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c069674-b18d-4206-9453-e1a3b7c031de admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:34:31.417956 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c069674-b18d-4206-9453-e1a3b7c031de admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:34:31.435573 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1030/2067] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Aug 6 01:34:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:34:31.437031 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8d322c-cd3c-48bd-9327-5cafd2c18d28 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:34:31.448699 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8d322c-cd3c-48bd-9327-5cafd2c18d28 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:34:31.453921 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292d637c-3bd5-4421-9bd2-f5aa5fb8b57b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:34:31.454166 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292d637c-3bd5-4421-9bd2-f5aa5fb8b57b admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:31.454447 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292d637c-3bd5-4421-9bd2-f5aa5fb8b57b admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:31.454726 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292d637c-3bd5-4421-9bd2-f5aa5fb8b57b admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:31.455402 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292d637c-3bd5-4421-9bd2-f5aa5fb8b57b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:34:31.457394 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292d637c-3bd5-4421-9bd2-f5aa5fb8b57b admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:34:31.488691 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1038/2068] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Aug 6 01:34:31 2022] POST /identity/v3/projects => generated 369 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:34:31.489959 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c069674-b18d-4206-9453-e1a3b7c031de admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:34:31.500571 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c069674-b18d-4206-9453-e1a3b7c031de admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:34:31.559192 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f55096-ef49-4c8a-b9d7-44bff962ac76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:34:31.559818 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f55096-ef49-4c8a-b9d7-44bff962ac76 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:31.560103 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f55096-ef49-4c8a-b9d7-44bff962ac76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:31.560382 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f55096-ef49-4c8a-b9d7-44bff962ac76 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:31.561082 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f55096-ef49-4c8a-b9d7-44bff962ac76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:34:31.562992 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f55096-ef49-4c8a-b9d7-44bff962ac76 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:34:31.587433 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1031/2069] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:34:31 2022] POST /identity/v3/users => generated 285 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:34:31.593248 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292d637c-3bd5-4421-9bd2-f5aa5fb8b57b admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:34:31.613078 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292d637c-3bd5-4421-9bd2-f5aa5fb8b57b admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:34:31.618961 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5784c5-e085-432b-aa2b-c91687f9e747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:34:31.619598 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5784c5-e085-432b-aa2b-c91687f9e747 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:31.619945 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5784c5-e085-432b-aa2b-c91687f9e747 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:31.620229 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5784c5-e085-432b-aa2b-c91687f9e747 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:31.621073 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5784c5-e085-432b-aa2b-c91687f9e747 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:34:31.622883 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5784c5-e085-432b-aa2b-c91687f9e747 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:34:31.637724 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1039/2070] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:34:31 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:34:31.642902 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f55096-ef49-4c8a-b9d7-44bff962ac76 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:34:31.658183 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f55096-ef49-4c8a-b9d7-44bff962ac76 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:34:31.668459 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bcc67b-335c-4323-9399-a007bc932ceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:34:31.670333 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bcc67b-335c-4323-9399-a007bc932ceb admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:31.670629 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bcc67b-335c-4323-9399-a007bc932ceb admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:31.671048 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bcc67b-335c-4323-9399-a007bc932ceb admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:31.671796 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bcc67b-335c-4323-9399-a007bc932ceb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:34:31.673640 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bcc67b-335c-4323-9399-a007bc932ceb admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:34:31.685580 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1032/2071] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:34:31 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:34:31.689666 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5784c5-e085-432b-aa2b-c91687f9e747 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:34:31.700911 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5784c5-e085-432b-aa2b-c91687f9e747 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:34:31.707373 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d32a691-b8a8-4ecb-8c31-ba683e13ca3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:34:31.708202 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d32a691-b8a8-4ecb-8c31-ba683e13ca3e admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:31.708557 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d32a691-b8a8-4ecb-8c31-ba683e13ca3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:31.708901 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d32a691-b8a8-4ecb-8c31-ba683e13ca3e admin admin] PATH_INFO: `/v3/projects/ec18238cf5d943f6a81040f664889bf9/users/f6daf3866ffe4f0dba7ec23e881576dd/roles/61b3b93d37a047e4bb8369b46fe20db9` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:31.726057 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d32a691-b8a8-4ecb-8c31-ba683e13ca3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec18238cf5d943f6a81040f664889bf9, user_id=f6daf3866ffe4f0dba7ec23e881576dd, role_id=61b3b93d37a047e4bb8369b46fe20db9)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:34:31.729247 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d32a691-b8a8-4ecb-8c31-ba683e13ca3e admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:34:31.744833 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1040/2072] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:34:31 2022] PUT /identity/v3/projects/ec18238cf5d943f6a81040f664889bf9/users/f6daf3866ffe4f0dba7ec23e881576dd/roles/61b3b93d37a047e4bb8369b46fe20db9 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:34:31.757715 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49461e62-e977-4c24-a4f1-b7ed79bad4ca None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:31.758313 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49461e62-e977-4c24-a4f1-b7ed79bad4ca None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:31.758901 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49461e62-e977-4c24-a4f1-b7ed79bad4ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:31.791415 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-49461e62-e977-4c24-a4f1-b7ed79bad4ca None None] MFA Rules not processed for user `f6daf3866ffe4f0dba7ec23e881576dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:34:31.793595 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-49461e62-e977-4c24-a4f1-b7ed79bad4ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:34:31.861273 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1033/2073] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:34:31 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:34:31.866049 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d32a691-b8a8-4ecb-8c31-ba683e13ca3e admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:34:31.893003 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:34:31.940197 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d32a691-b8a8-4ecb-8c31-ba683e13ca3e admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:34:31.945469 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9852b0dc-1be6-40a3-8fd8-5e15c7019d24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:34:31.946084 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9852b0dc-1be6-40a3-8fd8-5e15c7019d24 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:31.946370 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9852b0dc-1be6-40a3-8fd8-5e15c7019d24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:31.946648 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9852b0dc-1be6-40a3-8fd8-5e15c7019d24 admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:31.947328 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9852b0dc-1be6-40a3-8fd8-5e15c7019d24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:34:31.949246 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9852b0dc-1be6-40a3-8fd8-5e15c7019d24 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:34:31.980529 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1041/2074] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Aug 6 01:34:31 2022] POST /identity/v3/projects => generated 373 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:34:31.985220 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49461e62-e977-4c24-a4f1-b7ed79bad4ca None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:34:31.997843 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49461e62-e977-4c24-a4f1-b7ed79bad4ca None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:34:32.012014 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:34:32.059362 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4be6d8-d16b-4bef-8762-dc0a7ae11ff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:34:32.060019 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4be6d8-d16b-4bef-8762-dc0a7ae11ff0 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:32.060336 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4be6d8-d16b-4bef-8762-dc0a7ae11ff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:32.060614 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4be6d8-d16b-4bef-8762-dc0a7ae11ff0 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:32.061329 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4be6d8-d16b-4bef-8762-dc0a7ae11ff0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:34:32.063322 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4be6d8-d16b-4bef-8762-dc0a7ae11ff0 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:34:32.094953 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1034/2075] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:34:31 2022] POST /identity/v3/users => generated 293 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:34:32.101311 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9852b0dc-1be6-40a3-8fd8-5e15c7019d24 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:34:32.114204 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9852b0dc-1be6-40a3-8fd8-5e15c7019d24 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:34:32.119437 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ca393b-b412-4735-82f8-8621797e3ce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:34:32.120065 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ca393b-b412-4735-82f8-8621797e3ce3 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:32.120352 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ca393b-b412-4735-82f8-8621797e3ce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:32.120632 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ca393b-b412-4735-82f8-8621797e3ce3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:32.121371 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ca393b-b412-4735-82f8-8621797e3ce3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:34:32.123085 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ca393b-b412-4735-82f8-8621797e3ce3 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:34:32.134526 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1042/2076] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:34:32 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:34:32.138551 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4be6d8-d16b-4bef-8762-dc0a7ae11ff0 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:34:32.150685 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4be6d8-d16b-4bef-8762-dc0a7ae11ff0 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:34:32.155384 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fbc1b3-73ea-4bee-b7e4-dcec031e5060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:34:32.156055 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fbc1b3-73ea-4bee-b7e4-dcec031e5060 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:32.156341 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fbc1b3-73ea-4bee-b7e4-dcec031e5060 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:32.156621 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fbc1b3-73ea-4bee-b7e4-dcec031e5060 admin admin] PATH_INFO: `/v3/projects/59e455dbebbc4b55ad8fc98fceb6a867/users/4105b168d8744394a1f6198603018b4c/roles/4559b9a0837249f895a13c70096f8d38` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:32.173438 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fbc1b3-73ea-4bee-b7e4-dcec031e5060 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59e455dbebbc4b55ad8fc98fceb6a867, user_id=4105b168d8744394a1f6198603018b4c, role_id=4559b9a0837249f895a13c70096f8d38)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:34:32.176457 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fbc1b3-73ea-4bee-b7e4-dcec031e5060 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:34:32.194849 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1035/2077] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:34:32 2022] PUT /identity/v3/projects/59e455dbebbc4b55ad8fc98fceb6a867/users/4105b168d8744394a1f6198603018b4c/roles/4559b9a0837249f895a13c70096f8d38 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:34:32.199072 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ca393b-b412-4735-82f8-8621797e3ce3 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:34:32.214318 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ca393b-b412-4735-82f8-8621797e3ce3 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:34:32.317935 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0194a6ad-befa-40c6-a3cb-87b7c41b2f88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:34:32.318584 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0194a6ad-befa-40c6-a3cb-87b7c41b2f88 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:32.318868 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0194a6ad-befa-40c6-a3cb-87b7c41b2f88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:32.319144 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0194a6ad-befa-40c6-a3cb-87b7c41b2f88 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:32.319872 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0194a6ad-befa-40c6-a3cb-87b7c41b2f88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:34:32.321504 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0194a6ad-befa-40c6-a3cb-87b7c41b2f88 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:34:32.337020 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1043/2078] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:34:32 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:34:32.338284 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fbc1b3-73ea-4bee-b7e4-dcec031e5060 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:34:32.350410 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fbc1b3-73ea-4bee-b7e4-dcec031e5060 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:34:32.355586 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96b16a7-b247-4592-a87b-04c9f83f0178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:34:32.356303 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96b16a7-b247-4592-a87b-04c9f83f0178 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:32.356588 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96b16a7-b247-4592-a87b-04c9f83f0178 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:32.356866 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96b16a7-b247-4592-a87b-04c9f83f0178 admin admin] PATH_INFO: `/v3/projects/59e455dbebbc4b55ad8fc98fceb6a867/users/4105b168d8744394a1f6198603018b4c/roles/4559b9a0837249f895a13c70096f8d38` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:32.358895 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96b16a7-b247-4592-a87b-04c9f83f0178 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59e455dbebbc4b55ad8fc98fceb6a867, user_id=4105b168d8744394a1f6198603018b4c, role_id=4559b9a0837249f895a13c70096f8d38)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:34:32.361603 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96b16a7-b247-4592-a87b-04c9f83f0178 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:34:32.374525 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1036/2079] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:34:32 2022] PUT /identity/v3/projects/59e455dbebbc4b55ad8fc98fceb6a867/users/4105b168d8744394a1f6198603018b4c/roles/4559b9a0837249f895a13c70096f8d38 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:34:32.384882 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c674c9c-e759-42c7-a44b-18d122bbd2e0 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:32.385217 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c674c9c-e759-42c7-a44b-18d122bbd2e0 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:32.386381 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c674c9c-e759-42c7-a44b-18d122bbd2e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:32.417674 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-8c674c9c-e759-42c7-a44b-18d122bbd2e0 None None] MFA Rules not processed for user `4105b168d8744394a1f6198603018b4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:34:32.418912 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-8c674c9c-e759-42c7-a44b-18d122bbd2e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:34:32.495030 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1044/2080] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:34:32 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:34:32.505690 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96b16a7-b247-4592-a87b-04c9f83f0178 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:34:32.507437 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-a96b16a7-b247-4592-a87b-04c9f83f0178 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:34:32.564935 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96b16a7-b247-4592-a87b-04c9f83f0178 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:34:32.568624 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d02260-f076-453b-b373-9c28d6cfe7e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1be1e6173e374a60b9e1fbf51a1ab571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:34:32.569274 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d02260-f076-453b-b373-9c28d6cfe7e6 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:32.569577 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d02260-f076-453b-b373-9c28d6cfe7e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:32.569826 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d02260-f076-453b-b373-9c28d6cfe7e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:32.626411 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d02260-f076-453b-b373-9c28d6cfe7e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:34:32.628389 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d02260-f076-453b-b373-9c28d6cfe7e6 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:34:32.639914 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1037/2081] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:34:32 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:34:33.526810 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c674c9c-e759-42c7-a44b-18d122bbd2e0 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:34:33.528260 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-8c674c9c-e759-42c7-a44b-18d122bbd2e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:34:33.545730 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:34:33.592507 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c674c9c-e759-42c7-a44b-18d122bbd2e0 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:34:33.596955 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8c3302-6dfb-442c-9817-fa4e30d99948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:34:33.597612 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8c3302-6dfb-442c-9817-fa4e30d99948 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:33.597893 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8c3302-6dfb-442c-9817-fa4e30d99948 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:33.598162 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8c3302-6dfb-442c-9817-fa4e30d99948 admin admin] PATH_INFO: `/v3/users/8f51517b1e95427d9c6649582c253d41` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:33.599330 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8c3302-6dfb-442c-9817-fa4e30d99948 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f51517b1e95427d9c6649582c253d41)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:34:33.601335 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8c3302-6dfb-442c-9817-fa4e30d99948 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:34:33.647930 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-fb8c3302-6dfb-442c-9817-fa4e30d99948 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f51517b1e95427d9c6649582c253d41'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:33.654837 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-fb8c3302-6dfb-442c-9817-fa4e30d99948 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f51517b1e95427d9c6649582c253d41'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:33.655507 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-fb8c3302-6dfb-442c-9817-fa4e30d99948 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f51517b1e95427d9c6649582c253d41'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:33.656114 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-fb8c3302-6dfb-442c-9817-fa4e30d99948 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f51517b1e95427d9c6649582c253d41'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:33.667732 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-fb8c3302-6dfb-442c-9817-fa4e30d99948 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f51517b1e95427d9c6649582c253d41'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:33.681307 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1045/2082] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Aug 6 01:34:33 2022] DELETE /identity/v3/users/8f51517b1e95427d9c6649582c253d41 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:34:33.685733 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e16efe-fcf6-4270-b09b-97816f49bb68 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:33.686188 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e16efe-fcf6-4270-b09b-97816f49bb68 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:33.686287 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e16efe-fcf6-4270-b09b-97816f49bb68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:33.705926 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-09e16efe-fcf6-4270-b09b-97816f49bb68 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:34:33.707133 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-09e16efe-fcf6-4270-b09b-97816f49bb68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:34:33.725026 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:34:33.766058 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1038/2083] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:34:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:34:33.777587 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8c3302-6dfb-442c-9817-fa4e30d99948 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:34:33.780355 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-fb8c3302-6dfb-442c-9817-fa4e30d99948 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:34:33.856125 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8c3302-6dfb-442c-9817-fa4e30d99948 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:34:33.860408 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9ffe51-5270-41c7-89a6-979a10a8b6bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:34:33.861139 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9ffe51-5270-41c7-89a6-979a10a8b6bb service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:33.861382 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9ffe51-5270-41c7-89a6-979a10a8b6bb service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:33.861662 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9ffe51-5270-41c7-89a6-979a10a8b6bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:33.872484 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9ffe51-5270-41c7-89a6-979a10a8b6bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:34:33.874037 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9ffe51-5270-41c7-89a6-979a10a8b6bb service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:34:33.885485 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1046/2084] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:34:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:34:34.295427 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e16efe-fcf6-4270-b09b-97816f49bb68 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:34:34.297534 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-09e16efe-fcf6-4270-b09b-97816f49bb68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:34:34.311233 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e16efe-fcf6-4270-b09b-97816f49bb68 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:34:34.318760 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d33233-b7cd-417d-92a0-8bee367be456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:34:34.319704 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d33233-b7cd-417d-92a0-8bee367be456 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:34.319821 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d33233-b7cd-417d-92a0-8bee367be456 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:34.331307 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d33233-b7cd-417d-92a0-8bee367be456 admin admin] PATH_INFO: `/v3/projects/d681586de5344700900aa8d0cd9d9b6d` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:34.332765 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d33233-b7cd-417d-92a0-8bee367be456 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d681586de5344700900aa8d0cd9d9b6d)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:34:34.334779 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d33233-b7cd-417d-92a0-8bee367be456 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:34:34.392203 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-b5d33233-b7cd-417d-92a0-8bee367be456 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd681586de5344700900aa8d0cd9d9b6d', 'request_id': 'req-b5d33233-b7cd-417d-92a0-8bee367be456'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:34.400620 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-b5d33233-b7cd-417d-92a0-8bee367be456 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd681586de5344700900aa8d0cd9d9b6d', 'request_id': 'req-b5d33233-b7cd-417d-92a0-8bee367be456'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:34.407442 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-b5d33233-b7cd-417d-92a0-8bee367be456 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd681586de5344700900aa8d0cd9d9b6d', 'request_id': 'req-b5d33233-b7cd-417d-92a0-8bee367be456'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:34.423015 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-b5d33233-b7cd-417d-92a0-8bee367be456 admin admin] The token cache is being invalidate because project d681586de5344700900aa8d0cd9d9b6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:34:34.423368 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-b5d33233-b7cd-417d-92a0-8bee367be456 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:34.424179 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-b5d33233-b7cd-417d-92a0-8bee367be456 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:34.427894 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1039/2085] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Aug 6 01:34:34 2022] DELETE /identity/v3/projects/d681586de5344700900aa8d0cd9d9b6d => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:34:34.431945 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef22305-2292-4c45-9bce-d4e86c09b246 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:34.432377 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef22305-2292-4c45-9bce-d4e86c09b246 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:34.432731 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef22305-2292-4c45-9bce-d4e86c09b246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:34.474728 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-eef22305-2292-4c45-9bce-d4e86c09b246 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:34:34.476219 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-eef22305-2292-4c45-9bce-d4e86c09b246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:34:34.521382 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:34:34.583019 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1047/2086] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:34:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:34:34.651996 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3410a63-f35b-40f9-8bda-22ee611cebbe None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:34.652296 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3410a63-f35b-40f9-8bda-22ee611cebbe None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:34.652470 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3410a63-f35b-40f9-8bda-22ee611cebbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:34.673152 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-a3410a63-f35b-40f9-8bda-22ee611cebbe None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:34:34.674363 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-a3410a63-f35b-40f9-8bda-22ee611cebbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:34:34.683043 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1040/2087] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:34:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:34:34.687198 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef22305-2292-4c45-9bce-d4e86c09b246 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:34:34.698033 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef22305-2292-4c45-9bce-d4e86c09b246 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:34:34.702805 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bbead8-3d20-43bc-ade3-5d02df82bcde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:34:34.703425 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bbead8-3d20-43bc-ade3-5d02df82bcde admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:34.703709 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bbead8-3d20-43bc-ade3-5d02df82bcde admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:34.704017 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bbead8-3d20-43bc-ade3-5d02df82bcde admin admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:34.704723 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bbead8-3d20-43bc-ade3-5d02df82bcde admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:34:34.705031 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bbead8-3d20-43bc-ade3-5d02df82bcde admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:34:34.706661 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bbead8-3d20-43bc-ade3-5d02df82bcde admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:34:34.718532 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1048/2088] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Aug 6 01:34:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:34:34.723433 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3410a63-f35b-40f9-8bda-22ee611cebbe None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:34:34.732857 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3410a63-f35b-40f9-8bda-22ee611cebbe None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:34:34.738175 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9f35b2-78e3-4a71-b8d1-4d3a196fa27e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:34:34.738783 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9f35b2-78e3-4a71-b8d1-4d3a196fa27e admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:34.739064 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9f35b2-78e3-4a71-b8d1-4d3a196fa27e admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:34.739333 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9f35b2-78e3-4a71-b8d1-4d3a196fa27e admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:34.740243 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9f35b2-78e3-4a71-b8d1-4d3a196fa27e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:34:34.742183 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9f35b2-78e3-4a71-b8d1-4d3a196fa27e admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:34:34.768180 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1041/2089] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Aug 6 01:34:34 2022] POST /identity/v3/projects => generated 371 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:34:34.772592 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bbead8-3d20-43bc-ade3-5d02df82bcde admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:34:34.782525 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bbead8-3d20-43bc-ade3-5d02df82bcde admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:34:34.840908 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b852c69-9c6b-4ec0-abf7-649ec2f50af3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:34:34.841555 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b852c69-9c6b-4ec0-abf7-649ec2f50af3 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:34.841858 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b852c69-9c6b-4ec0-abf7-649ec2f50af3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:34.842127 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b852c69-9c6b-4ec0-abf7-649ec2f50af3 admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:34.842766 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b852c69-9c6b-4ec0-abf7-649ec2f50af3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:34:34.845097 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b852c69-9c6b-4ec0-abf7-649ec2f50af3 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:34:34.867593 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1049/2090] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:34:34 2022] POST /identity/v3/users => generated 286 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:34:34.871658 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9f35b2-78e3-4a71-b8d1-4d3a196fa27e admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:34:34.882708 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9f35b2-78e3-4a71-b8d1-4d3a196fa27e admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:34:34.887299 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40897f57-abc6-478e-bfce-5453ca800272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:34:34.887905 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40897f57-abc6-478e-bfce-5453ca800272 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:34.888186 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40897f57-abc6-478e-bfce-5453ca800272 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:34.888462 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40897f57-abc6-478e-bfce-5453ca800272 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:34.889176 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40897f57-abc6-478e-bfce-5453ca800272 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:34:34.890747 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40897f57-abc6-478e-bfce-5453ca800272 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:34:34.902666 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1042/2091] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:34:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:34:34.906641 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b852c69-9c6b-4ec0-abf7-649ec2f50af3 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:34:34.917231 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b852c69-9c6b-4ec0-abf7-649ec2f50af3 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:34:34.921904 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0221ccdb-3c11-4bf2-9e1d-983a3a06cc9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:34:34.922520 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0221ccdb-3c11-4bf2-9e1d-983a3a06cc9c admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:34.922801 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0221ccdb-3c11-4bf2-9e1d-983a3a06cc9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:34.923075 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0221ccdb-3c11-4bf2-9e1d-983a3a06cc9c admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:34.923823 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0221ccdb-3c11-4bf2-9e1d-983a3a06cc9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:34:34.926645 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0221ccdb-3c11-4bf2-9e1d-983a3a06cc9c admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:34:34.937436 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1050/2092] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:34:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:34:34.942293 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40897f57-abc6-478e-bfce-5453ca800272 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:34:34.952234 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40897f57-abc6-478e-bfce-5453ca800272 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:34:34.956910 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75c823b-b78f-4125-b29d-ce5d0ee4efb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:34:34.957666 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75c823b-b78f-4125-b29d-ce5d0ee4efb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:34.957948 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75c823b-b78f-4125-b29d-ce5d0ee4efb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:34.958222 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75c823b-b78f-4125-b29d-ce5d0ee4efb4 admin admin] PATH_INFO: `/v3/projects/722d0293c88d4711a7252d01bb91621d/users/cbb2fa7d4e5b42388cb4a97ec3495d51/roles/61b3b93d37a047e4bb8369b46fe20db9` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:34.973285 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75c823b-b78f-4125-b29d-ce5d0ee4efb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=722d0293c88d4711a7252d01bb91621d, user_id=cbb2fa7d4e5b42388cb4a97ec3495d51, role_id=61b3b93d37a047e4bb8369b46fe20db9)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:34:34.975944 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75c823b-b78f-4125-b29d-ce5d0ee4efb4 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:34:34.991160 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1043/2093] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:34:34 2022] PUT /identity/v3/projects/722d0293c88d4711a7252d01bb91621d/users/cbb2fa7d4e5b42388cb4a97ec3495d51/roles/61b3b93d37a047e4bb8369b46fe20db9 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:34:35.001835 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dea2e29-390d-4654-ac31-79b67e2b1540 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:35.002401 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dea2e29-390d-4654-ac31-79b67e2b1540 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:35.002685 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dea2e29-390d-4654-ac31-79b67e2b1540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:35.023549 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-4dea2e29-390d-4654-ac31-79b67e2b1540 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:34:35.024760 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-4dea2e29-390d-4654-ac31-79b67e2b1540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:34:35.084104 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1051/2094] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:34:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:34:35.093744 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75c823b-b78f-4125-b29d-ce5d0ee4efb4 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:34:35.095447 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-d75c823b-b78f-4125-b29d-ce5d0ee4efb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:34:35.144460 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:34:35.157626 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75c823b-b78f-4125-b29d-ce5d0ee4efb4 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:34:35.161615 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca02680a-7e5b-42d9-8246-5622cc61195f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:34:35.162239 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca02680a-7e5b-42d9-8246-5622cc61195f service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:35.162522 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca02680a-7e5b-42d9-8246-5622cc61195f service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:35.162801 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca02680a-7e5b-42d9-8246-5622cc61195f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:35.172206 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca02680a-7e5b-42d9-8246-5622cc61195f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:34:35.173669 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca02680a-7e5b-42d9-8246-5622cc61195f service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:34:35.184411 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1044/2095] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:34:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:34:35.291425 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dea2e29-390d-4654-ac31-79b67e2b1540 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:34:35.292850 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-4dea2e29-390d-4654-ac31-79b67e2b1540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:34:35.307279 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dea2e29-390d-4654-ac31-79b67e2b1540 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:34:35.311896 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69beebb-90c8-40af-946f-c7e6412f5525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:34:35.312965 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69beebb-90c8-40af-946f-c7e6412f5525 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:35.312965 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69beebb-90c8-40af-946f-c7e6412f5525 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:35.313212 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69beebb-90c8-40af-946f-c7e6412f5525 admin admin] PATH_INFO: `/v3/users/f6daf3866ffe4f0dba7ec23e881576dd` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:35.314271 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69beebb-90c8-40af-946f-c7e6412f5525 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6daf3866ffe4f0dba7ec23e881576dd)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:34:35.316796 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69beebb-90c8-40af-946f-c7e6412f5525 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:34:35.371661 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:34:35.385261 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-b69beebb-90c8-40af-946f-c7e6412f5525 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6daf3866ffe4f0dba7ec23e881576dd'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:35.431935 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-b69beebb-90c8-40af-946f-c7e6412f5525 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6daf3866ffe4f0dba7ec23e881576dd'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:35.433296 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-b69beebb-90c8-40af-946f-c7e6412f5525 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6daf3866ffe4f0dba7ec23e881576dd'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:35.436081 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-b69beebb-90c8-40af-946f-c7e6412f5525 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6daf3866ffe4f0dba7ec23e881576dd'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:35.472743 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-b69beebb-90c8-40af-946f-c7e6412f5525 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6daf3866ffe4f0dba7ec23e881576dd'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:35.494621 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1052/2096] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Aug 6 01:34:35 2022] DELETE /identity/v3/users/f6daf3866ffe4f0dba7ec23e881576dd => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:34:35.499562 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49590571-974c-48bf-98a9-4bb45ae0a5d8 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:35.500071 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49590571-974c-48bf-98a9-4bb45ae0a5d8 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:35.500566 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49590571-974c-48bf-98a9-4bb45ae0a5d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:35.558770 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-49590571-974c-48bf-98a9-4bb45ae0a5d8 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:34:35.560194 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-49590571-974c-48bf-98a9-4bb45ae0a5d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:34:35.681230 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1045/2097] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:34:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:34:35.686139 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69beebb-90c8-40af-946f-c7e6412f5525 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:34:35.688214 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-b69beebb-90c8-40af-946f-c7e6412f5525 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:34:35.780507 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69beebb-90c8-40af-946f-c7e6412f5525 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:34:35.785268 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abfbb5f-88a3-432f-9981-6af44bde9628 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:34:35.785953 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abfbb5f-88a3-432f-9981-6af44bde9628 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:35.786238 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abfbb5f-88a3-432f-9981-6af44bde9628 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:35.786513 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abfbb5f-88a3-432f-9981-6af44bde9628 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:35.796319 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abfbb5f-88a3-432f-9981-6af44bde9628 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:34:35.798754 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abfbb5f-88a3-432f-9981-6af44bde9628 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:34:35.813558 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1053/2098] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:34:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 01:34:36.150347 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747dcd3b-d59a-4551-8845-5eff5992ae50 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:36.150965 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747dcd3b-d59a-4551-8845-5eff5992ae50 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:36.150965 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747dcd3b-d59a-4551-8845-5eff5992ae50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:36.175684 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-747dcd3b-d59a-4551-8845-5eff5992ae50 None None] MFA Rules not processed for user `3f081e5336f74e21903f11df0b9d95b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:34:36.183700 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-747dcd3b-d59a-4551-8845-5eff5992ae50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:34:36.241305 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abfbb5f-88a3-432f-9981-6af44bde9628 service neutron] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:34:36.242877 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-7abfbb5f-88a3-432f-9981-6af44bde9628 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:34:36.258918 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1046/2099] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:34:36 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:34:36.281071 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abfbb5f-88a3-432f-9981-6af44bde9628 service neutron] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:34:36.305883 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afbf206-a0d6-433d-8af0-7141d8c0fd96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:34:36.306690 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afbf206-a0d6-433d-8af0-7141d8c0fd96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:36.307085 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afbf206-a0d6-433d-8af0-7141d8c0fd96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:36.307468 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afbf206-a0d6-433d-8af0-7141d8c0fd96 admin admin] PATH_INFO: `/v3/projects/ec18238cf5d943f6a81040f664889bf9` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:36.308851 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afbf206-a0d6-433d-8af0-7141d8c0fd96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec18238cf5d943f6a81040f664889bf9)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:34:36.310933 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afbf206-a0d6-433d-8af0-7141d8c0fd96 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:34:36.388866 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-0afbf206-a0d6-433d-8af0-7141d8c0fd96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec18238cf5d943f6a81040f664889bf9', 'request_id': 'req-0afbf206-a0d6-433d-8af0-7141d8c0fd96'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:36.404336 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-0afbf206-a0d6-433d-8af0-7141d8c0fd96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec18238cf5d943f6a81040f664889bf9', 'request_id': 'req-0afbf206-a0d6-433d-8af0-7141d8c0fd96'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:36.415316 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-0afbf206-a0d6-433d-8af0-7141d8c0fd96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec18238cf5d943f6a81040f664889bf9', 'request_id': 'req-0afbf206-a0d6-433d-8af0-7141d8c0fd96'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:36.431083 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-0afbf206-a0d6-433d-8af0-7141d8c0fd96 admin admin] The token cache is being invalidate because project ec18238cf5d943f6a81040f664889bf9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:34:36.431375 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-0afbf206-a0d6-433d-8af0-7141d8c0fd96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:36.431996 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-0afbf206-a0d6-433d-8af0-7141d8c0fd96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:36.433707 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1054/2100] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Aug 6 01:34:36 2022] DELETE /identity/v3/projects/ec18238cf5d943f6a81040f664889bf9 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:34:36.437724 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747dcd3b-d59a-4551-8845-5eff5992ae50 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:34:36.454297 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747dcd3b-d59a-4551-8845-5eff5992ae50 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:34:36.551697 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cd9990-0b5b-410a-847b-534d0a6e8e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:34:36.553042 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cd9990-0b5b-410a-847b-534d0a6e8e16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:36.553510 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cd9990-0b5b-410a-847b-534d0a6e8e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:36.553941 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cd9990-0b5b-410a-847b-534d0a6e8e16 admin admin] PATH_INFO: `/v3/users/4105b168d8744394a1f6198603018b4c` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:36.556879 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cd9990-0b5b-410a-847b-534d0a6e8e16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4105b168d8744394a1f6198603018b4c)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:34:36.559966 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cd9990-0b5b-410a-847b-534d0a6e8e16 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:34:36.640538 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:34:36.656173 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-a0cd9990-0b5b-410a-847b-534d0a6e8e16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4105b168d8744394a1f6198603018b4c'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:36.657093 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-a0cd9990-0b5b-410a-847b-534d0a6e8e16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4105b168d8744394a1f6198603018b4c'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:36.658069 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-a0cd9990-0b5b-410a-847b-534d0a6e8e16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4105b168d8744394a1f6198603018b4c'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:36.672769 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-a0cd9990-0b5b-410a-847b-534d0a6e8e16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4105b168d8744394a1f6198603018b4c'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:36.682318 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-a0cd9990-0b5b-410a-847b-534d0a6e8e16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4105b168d8744394a1f6198603018b4c'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:36.721864 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1047/2101] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Aug 6 01:34:36 2022] DELETE /identity/v3/users/4105b168d8744394a1f6198603018b4c => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:34:37.538146 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afbf206-a0d6-433d-8af0-7141d8c0fd96 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:34:37.540565 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-0afbf206-a0d6-433d-8af0-7141d8c0fd96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:34:37.585634 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afbf206-a0d6-433d-8af0-7141d8c0fd96 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:34:37.654499 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:34:37.693405 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67796205-9b80-492e-9865-859313bda397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:34:37.694171 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67796205-9b80-492e-9865-859313bda397 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:37.694526 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67796205-9b80-492e-9865-859313bda397 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:37.694873 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67796205-9b80-492e-9865-859313bda397 admin admin] PATH_INFO: `/v3/projects/59e455dbebbc4b55ad8fc98fceb6a867` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:37.696175 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67796205-9b80-492e-9865-859313bda397 admin admin] RBAC: Authorizing `identity:delete_project(project_id=59e455dbebbc4b55ad8fc98fceb6a867)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:34:37.699591 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67796205-9b80-492e-9865-859313bda397 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:34:37.763216 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-67796205-9b80-492e-9865-859313bda397 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59e455dbebbc4b55ad8fc98fceb6a867', 'request_id': 'req-67796205-9b80-492e-9865-859313bda397'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:37.772410 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-67796205-9b80-492e-9865-859313bda397 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59e455dbebbc4b55ad8fc98fceb6a867', 'request_id': 'req-67796205-9b80-492e-9865-859313bda397'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:37.780013 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-67796205-9b80-492e-9865-859313bda397 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59e455dbebbc4b55ad8fc98fceb6a867', 'request_id': 'req-67796205-9b80-492e-9865-859313bda397'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:37.797294 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-67796205-9b80-492e-9865-859313bda397 admin admin] The token cache is being invalidate because project 59e455dbebbc4b55ad8fc98fceb6a867 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:34:37.797845 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-67796205-9b80-492e-9865-859313bda397 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:37.798668 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-67796205-9b80-492e-9865-859313bda397 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:37.806361 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1055/2102] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Aug 6 01:34:37 2022] DELETE /identity/v3/projects/59e455dbebbc4b55ad8fc98fceb6a867 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:34:37.807828 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865c1d05-df34-4bbc-9999-42df0b926bb6 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:37.808311 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865c1d05-df34-4bbc-9999-42df0b926bb6 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:37.808426 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865c1d05-df34-4bbc-9999-42df0b926bb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:37.840105 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-865c1d05-df34-4bbc-9999-42df0b926bb6 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:34:37.841734 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-865c1d05-df34-4bbc-9999-42df0b926bb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:34:37.940677 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1048/2103] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:34:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:34:37.948882 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aadf0f-85dc-40c4-ad33-746a7dcbf6b1 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:37.949467 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aadf0f-85dc-40c4-ad33-746a7dcbf6b1 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:37.950078 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aadf0f-85dc-40c4-ad33-746a7dcbf6b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:37.979445 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-56aadf0f-85dc-40c4-ad33-746a7dcbf6b1 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:34:37.980977 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-56aadf0f-85dc-40c4-ad33-746a7dcbf6b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:34:37.997430 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865c1d05-df34-4bbc-9999-42df0b926bb6 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:34:37.998247 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1056/2104] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:34:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:34:38.016919 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865c1d05-df34-4bbc-9999-42df0b926bb6 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:34:38.022160 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b82912-be5b-43ee-a697-c8ed02616821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:34:38.022802 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b82912-be5b-43ee-a697-c8ed02616821 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:38.023083 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b82912-be5b-43ee-a697-c8ed02616821 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:38.023361 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b82912-be5b-43ee-a697-c8ed02616821 admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:38.024132 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b82912-be5b-43ee-a697-c8ed02616821 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:34:38.024449 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b82912-be5b-43ee-a697-c8ed02616821 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:34:38.026196 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b82912-be5b-43ee-a697-c8ed02616821 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:34:38.046592 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1049/2105] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Aug 6 01:34:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:34:38.052810 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56aadf0f-85dc-40c4-ad33-746a7dcbf6b1 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:34:38.074269 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56aadf0f-85dc-40c4-ad33-746a7dcbf6b1 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:34:38.089071 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c4ed3c-9a64-428d-b5b0-a7c3c58b156a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:34:38.089071 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c4ed3c-9a64-428d-b5b0-a7c3c58b156a admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:38.089071 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c4ed3c-9a64-428d-b5b0-a7c3c58b156a admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:38.089071 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c4ed3c-9a64-428d-b5b0-a7c3c58b156a admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:38.089700 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c4ed3c-9a64-428d-b5b0-a7c3c58b156a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:34:38.089700 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c4ed3c-9a64-428d-b5b0-a7c3c58b156a admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:34:38.126494 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1057/2106] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Aug 6 01:34:38 2022] POST /identity/v3/projects => generated 375 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:34:38.127951 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b82912-be5b-43ee-a697-c8ed02616821 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:34:38.141285 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b82912-be5b-43ee-a697-c8ed02616821 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:34:38.234336 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08455301-8dba-4fd6-bd0f-34f00a22ec05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:34:38.235573 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08455301-8dba-4fd6-bd0f-34f00a22ec05 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:38.236012 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08455301-8dba-4fd6-bd0f-34f00a22ec05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:38.236385 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08455301-8dba-4fd6-bd0f-34f00a22ec05 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:38.240089 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08455301-8dba-4fd6-bd0f-34f00a22ec05 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:34:38.242810 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08455301-8dba-4fd6-bd0f-34f00a22ec05 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:34:38.285969 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1050/2107] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:34:38 2022] POST /identity/v3/users => generated 288 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:34:38.287405 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c4ed3c-9a64-428d-b5b0-a7c3c58b156a admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:34:38.303501 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c4ed3c-9a64-428d-b5b0-a7c3c58b156a admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:34:38.313006 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ba6e94-4590-49ab-8b78-e1cd31dbf8ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:34:38.314593 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ba6e94-4590-49ab-8b78-e1cd31dbf8ac admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:38.315640 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ba6e94-4590-49ab-8b78-e1cd31dbf8ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:38.318139 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ba6e94-4590-49ab-8b78-e1cd31dbf8ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:38.320234 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ba6e94-4590-49ab-8b78-e1cd31dbf8ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:34:38.323016 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ba6e94-4590-49ab-8b78-e1cd31dbf8ac admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:34:38.346693 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1058/2108] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:34:38 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:34:38.350681 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08455301-8dba-4fd6-bd0f-34f00a22ec05 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:34:38.364453 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08455301-8dba-4fd6-bd0f-34f00a22ec05 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:34:38.377136 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10019a6-c6a8-466d-9a22-632a95e260ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:34:38.378348 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10019a6-c6a8-466d-9a22-632a95e260ac admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:38.378621 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10019a6-c6a8-466d-9a22-632a95e260ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:38.378897 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10019a6-c6a8-466d-9a22-632a95e260ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:38.380867 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10019a6-c6a8-466d-9a22-632a95e260ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:34:38.381542 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10019a6-c6a8-466d-9a22-632a95e260ac admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:34:38.402354 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1051/2109] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:34:38 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:34:38.403983 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ba6e94-4590-49ab-8b78-e1cd31dbf8ac admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:34:38.423877 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ba6e94-4590-49ab-8b78-e1cd31dbf8ac admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:34:38.429193 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a6032f-05b6-43e5-85a1-689948bf37d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:34:38.429925 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a6032f-05b6-43e5-85a1-689948bf37d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:38.430212 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a6032f-05b6-43e5-85a1-689948bf37d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:38.430490 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a6032f-05b6-43e5-85a1-689948bf37d1 admin admin] PATH_INFO: `/v3/projects/ae37cc98ea6d49feb79ed4b26bc82d23/users/a4f2d39d441545b5bb75f77c73eeedb7/roles/61b3b93d37a047e4bb8369b46fe20db9` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:38.453351 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a6032f-05b6-43e5-85a1-689948bf37d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae37cc98ea6d49feb79ed4b26bc82d23, user_id=a4f2d39d441545b5bb75f77c73eeedb7, role_id=61b3b93d37a047e4bb8369b46fe20db9)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:34:38.457869 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a6032f-05b6-43e5-85a1-689948bf37d1 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:34:38.483703 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1059/2110] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:34:38 2022] PUT /identity/v3/projects/ae37cc98ea6d49feb79ed4b26bc82d23/users/a4f2d39d441545b5bb75f77c73eeedb7/roles/61b3b93d37a047e4bb8369b46fe20db9 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:34:38.498725 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de498175-2175-47fe-a3e2-219aa38e4cb8 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:38.499265 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de498175-2175-47fe-a3e2-219aa38e4cb8 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:38.499716 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de498175-2175-47fe-a3e2-219aa38e4cb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:38.563236 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-de498175-2175-47fe-a3e2-219aa38e4cb8 None None] MFA Rules not processed for user `a4f2d39d441545b5bb75f77c73eeedb7`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:34:38.564659 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-de498175-2175-47fe-a3e2-219aa38e4cb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:34:38.593485 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:34:38.729749 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1052/2111] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:34:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:34:38.742804 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a6032f-05b6-43e5-85a1-689948bf37d1 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:34:38.744715 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-28a6032f-05b6-43e5-85a1-689948bf37d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:34:38.819329 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a6032f-05b6-43e5-85a1-689948bf37d1 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:34:38.823741 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6f249c-5d89-4a1a-a9cf-e5017c35a1c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1be1e6173e374a60b9e1fbf51a1ab571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:34:38.824457 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6f249c-5d89-4a1a-a9cf-e5017c35a1c2 service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:38.824807 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6f249c-5d89-4a1a-a9cf-e5017c35a1c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:38.825201 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6f249c-5d89-4a1a-a9cf-e5017c35a1c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:38.837652 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6f249c-5d89-4a1a-a9cf-e5017c35a1c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:34:38.841153 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6f249c-5d89-4a1a-a9cf-e5017c35a1c2 service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:34:38.852235 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1060/2112] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:34:38 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:34:43.041222 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de498175-2175-47fe-a3e2-219aa38e4cb8 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:34:43.150279 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de498175-2175-47fe-a3e2-219aa38e4cb8 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:34:43.157904 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa462a0-8038-42e5-b4dc-4a04ac72d672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:34:43.158576 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa462a0-8038-42e5-b4dc-4a04ac72d672 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:43.158907 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa462a0-8038-42e5-b4dc-4a04ac72d672 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:43.159193 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa462a0-8038-42e5-b4dc-4a04ac72d672 admin admin] PATH_INFO: `/v3/users/a4f2d39d441545b5bb75f77c73eeedb7` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:43.176188 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa462a0-8038-42e5-b4dc-4a04ac72d672 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4f2d39d441545b5bb75f77c73eeedb7)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:34:43.178217 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa462a0-8038-42e5-b4dc-4a04ac72d672 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:34:43.286736 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-aaa462a0-8038-42e5-b4dc-4a04ac72d672 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4f2d39d441545b5bb75f77c73eeedb7'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:43.287540 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-aaa462a0-8038-42e5-b4dc-4a04ac72d672 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4f2d39d441545b5bb75f77c73eeedb7'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:43.288709 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-aaa462a0-8038-42e5-b4dc-4a04ac72d672 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4f2d39d441545b5bb75f77c73eeedb7'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:43.321536 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-aaa462a0-8038-42e5-b4dc-4a04ac72d672 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4f2d39d441545b5bb75f77c73eeedb7'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:43.331748 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-aaa462a0-8038-42e5-b4dc-4a04ac72d672 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4f2d39d441545b5bb75f77c73eeedb7'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:43.378192 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1053/2113] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Aug 6 01:34:43 2022] DELETE /identity/v3/users/a4f2d39d441545b5bb75f77c73eeedb7 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:34:43.387840 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ccea2b-8afa-4f23-a70b-e6e60ce67281 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:43.388382 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ccea2b-8afa-4f23-a70b-e6e60ce67281 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:43.388484 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ccea2b-8afa-4f23-a70b-e6e60ce67281 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:43.455155 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-e2ccea2b-8afa-4f23-a70b-e6e60ce67281 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:34:43.456965 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-e2ccea2b-8afa-4f23-a70b-e6e60ce67281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:34:43.469842 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:34:43.538192 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1061/2114] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:34:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:34:43.563447 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa462a0-8038-42e5-b4dc-4a04ac72d672 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:34:43.566708 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-aaa462a0-8038-42e5-b4dc-4a04ac72d672 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:34:43.605213 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:34:43.641620 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa462a0-8038-42e5-b4dc-4a04ac72d672 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:34:43.650889 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8176eb14-4728-4acd-ae4b-398036ee214e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:34:43.658658 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8176eb14-4728-4acd-ae4b-398036ee214e service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:43.660052 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8176eb14-4728-4acd-ae4b-398036ee214e service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:43.660508 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8176eb14-4728-4acd-ae4b-398036ee214e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:43.667560 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31ebc57-d341-4495-8bec-aef68d3705d2 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:43.669173 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31ebc57-d341-4495-8bec-aef68d3705d2 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:43.669560 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31ebc57-d341-4495-8bec-aef68d3705d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:43.689487 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8176eb14-4728-4acd-ae4b-398036ee214e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:34:43.691611 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8176eb14-4728-4acd-ae4b-398036ee214e service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:34:43.711528 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1054/2115] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:34:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:34:43.753379 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-a31ebc57-d341-4495-8bec-aef68d3705d2 None None] MFA Rules not processed for user `cbb2fa7d4e5b42388cb4a97ec3495d51`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:34:43.755314 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-a31ebc57-d341-4495-8bec-aef68d3705d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:34:43.936697 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8176eb14-4728-4acd-ae4b-398036ee214e service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:34:43.938408 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-8176eb14-4728-4acd-ae4b-398036ee214e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:34:43.941224 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1062/2116] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:34:43 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:34:43.956310 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8176eb14-4728-4acd-ae4b-398036ee214e service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:34:43.974115 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08640729-9541-49f2-b43e-f786eaa78680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:34:43.974990 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08640729-9541-49f2-b43e-f786eaa78680 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:43.975436 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08640729-9541-49f2-b43e-f786eaa78680 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:43.975880 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08640729-9541-49f2-b43e-f786eaa78680 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:43.976721 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08640729-9541-49f2-b43e-f786eaa78680 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:34:43.979012 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08640729-9541-49f2-b43e-f786eaa78680 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:34:44.041544 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31ebc57-d341-4495-8bec-aef68d3705d2 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:34:44.052620 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1055/2117] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Aug 6 01:34:43 2022] POST /identity/v3/projects => generated 373 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:34:44.065270 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31ebc57-d341-4495-8bec-aef68d3705d2 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:34:44.209710 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bbaeff-1250-49e9-b916-a212912f49f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:34:44.210360 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bbaeff-1250-49e9-b916-a212912f49f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:44.210647 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bbaeff-1250-49e9-b916-a212912f49f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:44.210931 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bbaeff-1250-49e9-b916-a212912f49f0 admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:44.211619 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bbaeff-1250-49e9-b916-a212912f49f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:34:44.214902 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bbaeff-1250-49e9-b916-a212912f49f0 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:34:44.278379 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1063/2118] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:34:44 2022] POST /identity/v3/users => generated 293 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:34:44.281918 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08640729-9541-49f2-b43e-f786eaa78680 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:34:44.304685 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08640729-9541-49f2-b43e-f786eaa78680 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:34:44.320400 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631c4da9-6b7b-4de6-81a6-cd746d36259b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:34:44.321191 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631c4da9-6b7b-4de6-81a6-cd746d36259b admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:44.321498 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631c4da9-6b7b-4de6-81a6-cd746d36259b admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:44.321897 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631c4da9-6b7b-4de6-81a6-cd746d36259b admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:44.322722 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631c4da9-6b7b-4de6-81a6-cd746d36259b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:34:44.326936 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631c4da9-6b7b-4de6-81a6-cd746d36259b admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:34:44.353435 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1056/2119] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:34:44 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:34:44.358550 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bbaeff-1250-49e9-b916-a212912f49f0 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:34:44.375011 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bbaeff-1250-49e9-b916-a212912f49f0 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:34:44.390919 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188a6272-6678-442d-ae51-10dfc7ad6790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:34:44.391623 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188a6272-6678-442d-ae51-10dfc7ad6790 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:44.392600 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188a6272-6678-442d-ae51-10dfc7ad6790 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:44.393092 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188a6272-6678-442d-ae51-10dfc7ad6790 admin admin] PATH_INFO: `/v3/projects/9dfcddd9c01446688fe85ab203b609e8/users/ff7a95c0e54a40adb33ea6b152406554/roles/4559b9a0837249f895a13c70096f8d38` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:44.437742 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188a6272-6678-442d-ae51-10dfc7ad6790 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9dfcddd9c01446688fe85ab203b609e8, user_id=ff7a95c0e54a40adb33ea6b152406554, role_id=4559b9a0837249f895a13c70096f8d38)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:34:44.441716 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188a6272-6678-442d-ae51-10dfc7ad6790 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:34:44.466452 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631c4da9-6b7b-4de6-81a6-cd746d36259b admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:34:44.466640 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1064/2120] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:34:44 2022] PUT /identity/v3/projects/9dfcddd9c01446688fe85ab203b609e8/users/ff7a95c0e54a40adb33ea6b152406554/roles/4559b9a0837249f895a13c70096f8d38 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:34:44.478749 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631c4da9-6b7b-4de6-81a6-cd746d36259b admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:34:44.623127 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0451d3f-55fd-4bbe-9cbc-c619165d0f79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:34:44.626248 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0451d3f-55fd-4bbe-9cbc-c619165d0f79 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:44.627080 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0451d3f-55fd-4bbe-9cbc-c619165d0f79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:44.628402 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0451d3f-55fd-4bbe-9cbc-c619165d0f79 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:44.629260 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0451d3f-55fd-4bbe-9cbc-c619165d0f79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:34:44.632800 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0451d3f-55fd-4bbe-9cbc-c619165d0f79 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:34:44.684040 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188a6272-6678-442d-ae51-10dfc7ad6790 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:34:44.686060 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1057/2121] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:34:44 2022] GET /identity/v3/roles => generated 1420 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 01:34:44.707151 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188a6272-6678-442d-ae51-10dfc7ad6790 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:34:44.719409 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66f6cdc-6244-46a8-a789-3b63b2c4fe7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:34:44.722105 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66f6cdc-6244-46a8-a789-3b63b2c4fe7c admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:44.725302 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66f6cdc-6244-46a8-a789-3b63b2c4fe7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:44.725706 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66f6cdc-6244-46a8-a789-3b63b2c4fe7c admin admin] PATH_INFO: `/v3/projects/9dfcddd9c01446688fe85ab203b609e8/users/ff7a95c0e54a40adb33ea6b152406554/roles/4559b9a0837249f895a13c70096f8d38` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:44.730148 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66f6cdc-6244-46a8-a789-3b63b2c4fe7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9dfcddd9c01446688fe85ab203b609e8, user_id=ff7a95c0e54a40adb33ea6b152406554, role_id=4559b9a0837249f895a13c70096f8d38)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:34:44.737301 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66f6cdc-6244-46a8-a789-3b63b2c4fe7c admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:34:44.740407 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:34:44.764542 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1065/2122] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:34:44 2022] PUT /identity/v3/projects/9dfcddd9c01446688fe85ab203b609e8/users/ff7a95c0e54a40adb33ea6b152406554/roles/4559b9a0837249f895a13c70096f8d38 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:34:45.176729 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d920d462-8ad7-4d0e-9599-4367a75d6420 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:45.177261 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d920d462-8ad7-4d0e-9599-4367a75d6420 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:45.177487 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d920d462-8ad7-4d0e-9599-4367a75d6420 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:45.179060 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1058/2123] 10.222.0.26 () {54 vars in 921 bytes} [Sat Aug 6 01:34:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:34:45.213654 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb61638-55ce-4e89-bb27-c483bb0b6e86 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:45.214016 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb61638-55ce-4e89-bb27-c483bb0b6e86 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:45.214299 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb61638-55ce-4e89-bb27-c483bb0b6e86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:45.252272 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-ecb61638-55ce-4e89-bb27-c483bb0b6e86 None None] MFA Rules not processed for user `fa3b93ddd9464a2cba724d67b56b477a`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:34:45.252272 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-ecb61638-55ce-4e89-bb27-c483bb0b6e86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:34:45.323105 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d920d462-8ad7-4d0e-9599-4367a75d6420 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:34:45.324959 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-d920d462-8ad7-4d0e-9599-4367a75d6420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:34:45.447157 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1066/2124] 10.222.0.26 () {58 vars in 1018 bytes} [Sat Aug 6 01:34:45 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:34:45.450449 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d920d462-8ad7-4d0e-9599-4367a75d6420 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:34:45.456945 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b13fe5-93bc-438e-a22f-c1ce78fe04a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:34:45.457741 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b13fe5-93bc-438e-a22f-c1ce78fe04a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:45.458744 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b13fe5-93bc-438e-a22f-c1ce78fe04a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:45.459036 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b13fe5-93bc-438e-a22f-c1ce78fe04a7 admin admin] PATH_INFO: `/v3/users/5823f66f313843ddac68ff7a951321ae` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:45.460343 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b13fe5-93bc-438e-a22f-c1ce78fe04a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5823f66f313843ddac68ff7a951321ae)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:34:45.462468 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b13fe5-93bc-438e-a22f-c1ce78fe04a7 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:34:45.479077 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42051929-4bb9-4711-8646-e8ad62e089da None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:45.479428 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42051929-4bb9-4711-8646-e8ad62e089da None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:45.479740 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42051929-4bb9-4711-8646-e8ad62e089da None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:45.482260 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1067/2125] 10.222.0.26 () {54 vars in 961 bytes} [Sat Aug 6 01:34:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 01:34:45.489711 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285f2a25-04a3-4897-b1a7-9482e483f107 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:45.490302 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285f2a25-04a3-4897-b1a7-9482e483f107 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:45.490978 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285f2a25-04a3-4897-b1a7-9482e483f107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:45.532925 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-285f2a25-04a3-4897-b1a7-9482e483f107 None None] MFA Rules not processed for user `fa3b93ddd9464a2cba724d67b56b477a`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:34:45.533339 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:34:45.537944 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-285f2a25-04a3-4897-b1a7-9482e483f107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:34:45.552789 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1068/2126] 10.222.0.26 () {58 vars in 1058 bytes} [Sat Aug 6 01:34:45 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:34:45.576596 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-b3b13fe5-93bc-438e-a22f-c1ce78fe04a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5823f66f313843ddac68ff7a951321ae'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:45.578573 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285f2a25-04a3-4897-b1a7-9482e483f107 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:34:45.585610 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-b3b13fe5-93bc-438e-a22f-c1ce78fe04a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5823f66f313843ddac68ff7a951321ae'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:45.586656 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-b3b13fe5-93bc-438e-a22f-c1ce78fe04a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5823f66f313843ddac68ff7a951321ae'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:45.605582 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-b3b13fe5-93bc-438e-a22f-c1ce78fe04a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5823f66f313843ddac68ff7a951321ae'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:45.619345 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285f2a25-04a3-4897-b1a7-9482e483f107 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:34:45.623324 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-b3b13fe5-93bc-438e-a22f-c1ce78fe04a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5823f66f313843ddac68ff7a951321ae'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:45.628384 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766ff232-845a-40e6-a9b3-ec368ad5ddbd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa3b93ddd9464a2cba724d67b56b477a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:34:45.629419 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766ff232-845a-40e6-a9b3-ec368ad5ddbd service placement] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:45.629774 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766ff232-845a-40e6-a9b3-ec368ad5ddbd service placement] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:45.630056 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766ff232-845a-40e6-a9b3-ec368ad5ddbd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:45.632458 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-766ff232-845a-40e6-a9b3-ec368ad5ddbd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:34:45.643800 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1059/2127] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Aug 6 01:34:45 2022] DELETE /identity/v3/users/5823f66f313843ddac68ff7a951321ae => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:34:45.644714 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b13fe5-93bc-438e-a22f-c1ce78fe04a7 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:34:45.646277 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-b3b13fe5-93bc-438e-a22f-c1ce78fe04a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:34:45.662798 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766ff232-845a-40e6-a9b3-ec368ad5ddbd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:34:45.665798 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766ff232-845a-40e6-a9b3-ec368ad5ddbd service placement] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:34:45.695865 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1069/2128] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:34:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:34:45.778144 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b13fe5-93bc-438e-a22f-c1ce78fe04a7 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:34:45.785839 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c61b3b-28f8-43d6-9b32-86172edd6c9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:34:45.787223 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c61b3b-28f8-43d6-9b32-86172edd6c9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:45.787735 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c61b3b-28f8-43d6-9b32-86172edd6c9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:45.788425 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c61b3b-28f8-43d6-9b32-86172edd6c9e admin admin] PATH_INFO: `/v3/projects/ae37cc98ea6d49feb79ed4b26bc82d23` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:45.790240 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c61b3b-28f8-43d6-9b32-86172edd6c9e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae37cc98ea6d49feb79ed4b26bc82d23)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:34:45.792592 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c61b3b-28f8-43d6-9b32-86172edd6c9e admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:34:45.868232 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-f7c61b3b-28f8-43d6-9b32-86172edd6c9e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae37cc98ea6d49feb79ed4b26bc82d23', 'request_id': 'req-f7c61b3b-28f8-43d6-9b32-86172edd6c9e'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:45.878165 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-f7c61b3b-28f8-43d6-9b32-86172edd6c9e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae37cc98ea6d49feb79ed4b26bc82d23', 'request_id': 'req-f7c61b3b-28f8-43d6-9b32-86172edd6c9e'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:45.886450 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-f7c61b3b-28f8-43d6-9b32-86172edd6c9e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae37cc98ea6d49feb79ed4b26bc82d23', 'request_id': 'req-f7c61b3b-28f8-43d6-9b32-86172edd6c9e'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:45.915187 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-f7c61b3b-28f8-43d6-9b32-86172edd6c9e admin admin] The token cache is being invalidate because project ae37cc98ea6d49feb79ed4b26bc82d23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:34:45.915896 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-f7c61b3b-28f8-43d6-9b32-86172edd6c9e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:45.916599 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-f7c61b3b-28f8-43d6-9b32-86172edd6c9e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:45.918498 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1060/2129] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Aug 6 01:34:45 2022] DELETE /identity/v3/projects/ae37cc98ea6d49feb79ed4b26bc82d23 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:34:45.925455 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c6aa22-134e-4884-af36-f960ef2794f2 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:45.925455 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c6aa22-134e-4884-af36-f960ef2794f2 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:45.926631 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c6aa22-134e-4884-af36-f960ef2794f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:45.942628 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c61b3b-28f8-43d6-9b32-86172edd6c9e admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:34:45.947579 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-f7c61b3b-28f8-43d6-9b32-86172edd6c9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:34:45.975267 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-d7c6aa22-134e-4884-af36-f960ef2794f2 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:34:45.976484 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-d7c6aa22-134e-4884-af36-f960ef2794f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:34:46.101987 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:34:46.117922 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1070/2130] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:34:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:34:46.126444 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0911bd77-b627-4770-993f-701a43460540 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:46.127042 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0911bd77-b627-4770-993f-701a43460540 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:46.130287 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0911bd77-b627-4770-993f-701a43460540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:46.133184 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c61b3b-28f8-43d6-9b32-86172edd6c9e admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:34:46.146603 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75beab23-4974-47f5-bfa7-dbe0b6baff6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:34:46.149807 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75beab23-4974-47f5-bfa7-dbe0b6baff6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:46.150100 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75beab23-4974-47f5-bfa7-dbe0b6baff6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:46.150397 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75beab23-4974-47f5-bfa7-dbe0b6baff6e admin admin] PATH_INFO: `/v3/projects/b124d9abc46d4e85927cf1b40dd4a171` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:46.154377 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75beab23-4974-47f5-bfa7-dbe0b6baff6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b124d9abc46d4e85927cf1b40dd4a171)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:34:46.156594 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75beab23-4974-47f5-bfa7-dbe0b6baff6e admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:34:46.192662 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-0911bd77-b627-4770-993f-701a43460540 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:34:46.194181 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-0911bd77-b627-4770-993f-701a43460540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:34:46.211498 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1071/2131] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:34:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:34:46.215599 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0911bd77-b627-4770-993f-701a43460540 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:34:46.233025 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0911bd77-b627-4770-993f-701a43460540 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:34:46.244551 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a646070-133f-47c4-a8c4-7e8469558e00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:34:46.246432 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a646070-133f-47c4-a8c4-7e8469558e00 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:46.247569 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a646070-133f-47c4-a8c4-7e8469558e00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:46.247962 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a646070-133f-47c4-a8c4-7e8469558e00 admin admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:46.249652 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a646070-133f-47c4-a8c4-7e8469558e00 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:34:46.250159 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a646070-133f-47c4-a8c4-7e8469558e00 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:34:46.251603 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a646070-133f-47c4-a8c4-7e8469558e00 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:34:46.255182 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-75beab23-4974-47f5-bfa7-dbe0b6baff6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b124d9abc46d4e85927cf1b40dd4a171', 'request_id': 'req-75beab23-4974-47f5-bfa7-dbe0b6baff6e'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:46.262742 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-75beab23-4974-47f5-bfa7-dbe0b6baff6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b124d9abc46d4e85927cf1b40dd4a171', 'request_id': 'req-75beab23-4974-47f5-bfa7-dbe0b6baff6e'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:46.272007 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1072/2132] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Aug 6 01:34:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:34:46.274446 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-75beab23-4974-47f5-bfa7-dbe0b6baff6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b124d9abc46d4e85927cf1b40dd4a171', 'request_id': 'req-75beab23-4974-47f5-bfa7-dbe0b6baff6e'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:46.276297 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a646070-133f-47c4-a8c4-7e8469558e00 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:34:46.293331 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a646070-133f-47c4-a8c4-7e8469558e00 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:34:46.296194 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-75beab23-4974-47f5-bfa7-dbe0b6baff6e admin admin] The token cache is being invalidate because project b124d9abc46d4e85927cf1b40dd4a171 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:34:46.296503 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-75beab23-4974-47f5-bfa7-dbe0b6baff6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:46.297408 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-75beab23-4974-47f5-bfa7-dbe0b6baff6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:34:46.301159 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1061/2133] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Aug 6 01:34:45 2022] DELETE /identity/v3/projects/b124d9abc46d4e85927cf1b40dd4a171 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:34:46.306170 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3e0cc4-3b29-48f6-a39b-082fa5c569ee None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:46.306844 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3e0cc4-3b29-48f6-a39b-082fa5c569ee None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:46.307142 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3e0cc4-3b29-48f6-a39b-082fa5c569ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:46.349789 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-3c3e0cc4-3b29-48f6-a39b-082fa5c569ee None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:34:46.351385 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-3c3e0cc4-3b29-48f6-a39b-082fa5c569ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:34:46.389262 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:34:46.415148 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc3819b-f37b-47a8-aa42-0bf9d67804fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:34:46.416299 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc3819b-f37b-47a8-aa42-0bf9d67804fa admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:46.416828 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc3819b-f37b-47a8-aa42-0bf9d67804fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:46.417164 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc3819b-f37b-47a8-aa42-0bf9d67804fa admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:46.418273 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc3819b-f37b-47a8-aa42-0bf9d67804fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:34:46.420657 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc3819b-f37b-47a8-aa42-0bf9d67804fa admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:34:46.458378 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1062/2134] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:34:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:34:46.467990 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04ac804-af29-46b1-ab0c-54d3ca6d8a90 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:46.467990 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04ac804-af29-46b1-ab0c-54d3ca6d8a90 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:46.469726 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1073/2135] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Aug 6 01:34:46 2022] POST /identity/v3/projects => generated 367 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:34:46.470275 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04ac804-af29-46b1-ab0c-54d3ca6d8a90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:46.473973 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc3819b-f37b-47a8-aa42-0bf9d67804fa admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:34:46.476017 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-fbc3819b-f37b-47a8-aa42-0bf9d67804fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:34:46.496944 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-f04ac804-af29-46b1-ab0c-54d3ca6d8a90 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:34:46.498552 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-f04ac804-af29-46b1-ab0c-54d3ca6d8a90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:34:46.585941 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc3819b-f37b-47a8-aa42-0bf9d67804fa admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:34:46.591136 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1063/2136] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:34:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:34:46.591302 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dbcfe1-da79-4d78-8ff8-61919bfc8f7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:34:46.591302 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dbcfe1-da79-4d78-8ff8-61919bfc8f7e admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:46.591302 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dbcfe1-da79-4d78-8ff8-61919bfc8f7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:46.591302 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dbcfe1-da79-4d78-8ff8-61919bfc8f7e admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:46.591823 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dbcfe1-da79-4d78-8ff8-61919bfc8f7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:34:46.593565 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dbcfe1-da79-4d78-8ff8-61919bfc8f7e admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:34:46.594481 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04ac804-af29-46b1-ab0c-54d3ca6d8a90 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:34:46.606273 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04ac804-af29-46b1-ab0c-54d3ca6d8a90 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:34:46.613723 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a68b28f-2075-476e-a1cb-7ac1977d97c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:34:46.614470 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a68b28f-2075-476e-a1cb-7ac1977d97c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:46.614754 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a68b28f-2075-476e-a1cb-7ac1977d97c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:46.615460 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a68b28f-2075-476e-a1cb-7ac1977d97c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:46.616509 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a68b28f-2075-476e-a1cb-7ac1977d97c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:34:46.616830 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a68b28f-2075-476e-a1cb-7ac1977d97c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:34:46.618759 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a68b28f-2075-476e-a1cb-7ac1977d97c8 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:34:46.642716 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1074/2137] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:34:46 2022] POST /identity/v3/users => generated 284 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:34:46.647739 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:34:46.650058 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dbcfe1-da79-4d78-8ff8-61919bfc8f7e admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:34:46.657501 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1064/2138] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Aug 6 01:34:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:34:46.658637 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a68b28f-2075-476e-a1cb-7ac1977d97c8 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:34:46.664026 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dbcfe1-da79-4d78-8ff8-61919bfc8f7e admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:34:46.671146 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bd8b14-8566-4eb6-97b8-22e5f137c6dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:34:46.671998 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bd8b14-8566-4eb6-97b8-22e5f137c6dd admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:46.672283 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bd8b14-8566-4eb6-97b8-22e5f137c6dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:46.672557 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bd8b14-8566-4eb6-97b8-22e5f137c6dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:46.673433 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bd8b14-8566-4eb6-97b8-22e5f137c6dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:34:46.675633 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bd8b14-8566-4eb6-97b8-22e5f137c6dd admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:34:46.702653 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1075/2139] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:34:46 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:34:46.702854 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a68b28f-2075-476e-a1cb-7ac1977d97c8 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:34:46.705411 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bd8b14-8566-4eb6-97b8-22e5f137c6dd admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:34:46.718566 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d1874a-216c-4734-a1a9-29e9c395b5a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:34:46.718566 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d1874a-216c-4734-a1a9-29e9c395b5a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:46.718885 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d1874a-216c-4734-a1a9-29e9c395b5a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:46.719446 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d1874a-216c-4734-a1a9-29e9c395b5a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:46.721499 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d1874a-216c-4734-a1a9-29e9c395b5a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:34:46.725767 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d1874a-216c-4734-a1a9-29e9c395b5a0 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:34:46.728821 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bd8b14-8566-4eb6-97b8-22e5f137c6dd admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:34:46.734106 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b75c7c6-de2c-44a7-b5ad-a2115e12cf12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:34:46.734751 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b75c7c6-de2c-44a7-b5ad-a2115e12cf12 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:46.735083 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b75c7c6-de2c-44a7-b5ad-a2115e12cf12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:46.735685 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b75c7c6-de2c-44a7-b5ad-a2115e12cf12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:46.736521 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b75c7c6-de2c-44a7-b5ad-a2115e12cf12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:34:46.739401 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b75c7c6-de2c-44a7-b5ad-a2115e12cf12 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:34:46.752173 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1076/2140] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:34:46 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:34:46.757269 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b75c7c6-de2c-44a7-b5ad-a2115e12cf12 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:34:46.786471 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b75c7c6-de2c-44a7-b5ad-a2115e12cf12 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:34:46.791871 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d5f907-2e12-4033-8926-4898051c6774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:34:46.792835 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d5f907-2e12-4033-8926-4898051c6774 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:46.793499 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d5f907-2e12-4033-8926-4898051c6774 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:46.794214 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d5f907-2e12-4033-8926-4898051c6774 admin admin] PATH_INFO: `/v3/projects/1e7cfc49aa79444f8681a8be334a40bd/users/b9be4806b0b64d88bdb45733cec065d8/roles/61b3b93d37a047e4bb8369b46fe20db9` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:46.812195 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d5f907-2e12-4033-8926-4898051c6774 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e7cfc49aa79444f8681a8be334a40bd, user_id=b9be4806b0b64d88bdb45733cec065d8, role_id=61b3b93d37a047e4bb8369b46fe20db9)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:34:46.816400 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d5f907-2e12-4033-8926-4898051c6774 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:34:46.818968 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1065/2141] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Aug 6 01:34:46 2022] POST /identity/v3/projects => generated 383 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:34:46.824224 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d1874a-216c-4734-a1a9-29e9c395b5a0 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:34:46.850911 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1077/2142] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:34:46 2022] PUT /identity/v3/projects/1e7cfc49aa79444f8681a8be334a40bd/users/b9be4806b0b64d88bdb45733cec065d8/roles/61b3b93d37a047e4bb8369b46fe20db9 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:34:46.860962 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32243dad-89d9-4fd9-8aae-7b0e875f83b4 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:46.861282 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32243dad-89d9-4fd9-8aae-7b0e875f83b4 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:46.861677 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32243dad-89d9-4fd9-8aae-7b0e875f83b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:46.917255 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-32243dad-89d9-4fd9-8aae-7b0e875f83b4 None None] MFA Rules not processed for user `b9be4806b0b64d88bdb45733cec065d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:34:46.918576 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-32243dad-89d9-4fd9-8aae-7b0e875f83b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:34:46.943317 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d1874a-216c-4734-a1a9-29e9c395b5a0 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:34:47.034943 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1078/2143] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:34:46 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:34:47.044659 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32243dad-89d9-4fd9-8aae-7b0e875f83b4 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:34:47.048239 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-32243dad-89d9-4fd9-8aae-7b0e875f83b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:34:47.065952 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:34:47.077958 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12e3fc1-62f9-4737-bd80-1a6a4a031261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:34:47.085535 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12e3fc1-62f9-4737-bd80-1a6a4a031261 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:47.085891 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12e3fc1-62f9-4737-bd80-1a6a4a031261 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:47.086185 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12e3fc1-62f9-4737-bd80-1a6a4a031261 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:47.086915 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12e3fc1-62f9-4737-bd80-1a6a4a031261 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:34:47.100934 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12e3fc1-62f9-4737-bd80-1a6a4a031261 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:34:47.152705 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1066/2144] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:34:46 2022] POST /identity/v3/users => generated 292 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:34:47.157366 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12e3fc1-62f9-4737-bd80-1a6a4a031261 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:34:47.178738 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12e3fc1-62f9-4737-bd80-1a6a4a031261 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:34:47.187348 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32243dad-89d9-4fd9-8aae-7b0e875f83b4 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:34:47.192454 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade6e520-eaac-4c2a-84c7-845bfc4a4d34 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abb76a5c8ef74bf2b18c56074daaad58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:34:47.192690 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f74b1a8-976d-43b0-9b9b-57d0d34e6a4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:34:47.193296 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade6e520-eaac-4c2a-84c7-845bfc4a4d34 service glance] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:47.193552 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade6e520-eaac-4c2a-84c7-845bfc4a4d34 service glance] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:47.193697 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f74b1a8-976d-43b0-9b9b-57d0d34e6a4c admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:47.193914 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade6e520-eaac-4c2a-84c7-845bfc4a4d34 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:47.194398 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f74b1a8-976d-43b0-9b9b-57d0d34e6a4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:47.196422 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f74b1a8-976d-43b0-9b9b-57d0d34e6a4c admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:47.197478 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f74b1a8-976d-43b0-9b9b-57d0d34e6a4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:34:47.201759 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f74b1a8-976d-43b0-9b9b-57d0d34e6a4c admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:34:47.212826 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade6e520-eaac-4c2a-84c7-845bfc4a4d34 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:34:47.215147 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade6e520-eaac-4c2a-84c7-845bfc4a4d34 service glance] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:34:47.235896 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1079/2145] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:34:47 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 01:34:47.241091 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1067/2146] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:34:47 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:34:47.243160 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade6e520-eaac-4c2a-84c7-845bfc4a4d34 service glance] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:34:47.295719 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade6e520-eaac-4c2a-84c7-845bfc4a4d34 service glance] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:34:47.321944 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8661bbe-4e0a-4783-aca7-0dc7b4adcee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:34:47.323639 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8661bbe-4e0a-4783-aca7-0dc7b4adcee1 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:47.323934 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8661bbe-4e0a-4783-aca7-0dc7b4adcee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:47.324213 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8661bbe-4e0a-4783-aca7-0dc7b4adcee1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:47.324936 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8661bbe-4e0a-4783-aca7-0dc7b4adcee1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:34:47.326650 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8661bbe-4e0a-4783-aca7-0dc7b4adcee1 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:34:47.342796 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1080/2147] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:34:47 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:34:47.345586 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f74b1a8-976d-43b0-9b9b-57d0d34e6a4c admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:34:47.366548 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f74b1a8-976d-43b0-9b9b-57d0d34e6a4c admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:34:47.385871 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98308f5-0dff-42a4-ae55-543858506eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:34:47.386481 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98308f5-0dff-42a4-ae55-543858506eac admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:47.386762 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98308f5-0dff-42a4-ae55-543858506eac admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:47.387037 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98308f5-0dff-42a4-ae55-543858506eac admin admin] PATH_INFO: `/v3/projects/61190280a86846b5afd9b0d768f89713/users/2846129dffcd40abbd92eca26c271800/roles/61b3b93d37a047e4bb8369b46fe20db9` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:47.436128 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98308f5-0dff-42a4-ae55-543858506eac admin admin] RBAC: Authorizing `identity:create_grant(project_id=61190280a86846b5afd9b0d768f89713, user_id=2846129dffcd40abbd92eca26c271800, role_id=61b3b93d37a047e4bb8369b46fe20db9)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:34:47.440464 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98308f5-0dff-42a4-ae55-543858506eac admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:34:47.457461 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1068/2148] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:34:47 2022] PUT /identity/v3/projects/61190280a86846b5afd9b0d768f89713/users/2846129dffcd40abbd92eca26c271800/roles/61b3b93d37a047e4bb8369b46fe20db9 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:34:47.468030 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1470d2-a0fa-48be-a173-32b206018ae0 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:47.468331 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1470d2-a0fa-48be-a173-32b206018ae0 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:47.468626 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1470d2-a0fa-48be-a173-32b206018ae0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:47.525245 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-8c1470d2-a0fa-48be-a173-32b206018ae0 None None] MFA Rules not processed for user `2846129dffcd40abbd92eca26c271800`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:34:47.526469 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-8c1470d2-a0fa-48be-a173-32b206018ae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:34:47.707900 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1081/2149] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:34:47 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:34:47.951113 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98308f5-0dff-42a4-ae55-543858506eac admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:34:47.953399 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-a98308f5-0dff-42a4-ae55-543858506eac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:34:48.039602 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:34:48.110725 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98308f5-0dff-42a4-ae55-543858506eac admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:34:48.117729 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26284b69-ed03-4855-8f5d-b99ec2dcda8b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87adc1d5bc240e99f95f6595b350be4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:34:48.118381 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26284b69-ed03-4855-8f5d-b99ec2dcda8b service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:48.118745 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26284b69-ed03-4855-8f5d-b99ec2dcda8b service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:48.118978 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26284b69-ed03-4855-8f5d-b99ec2dcda8b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:48.141263 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26284b69-ed03-4855-8f5d-b99ec2dcda8b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:34:48.147300 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26284b69-ed03-4855-8f5d-b99ec2dcda8b service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:34:48.163809 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1069/2150] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:34:47 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:34:51.306208 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371b24a9-bc73-4b92-a4f9-7a162ccee264 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:51.307351 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371b24a9-bc73-4b92-a4f9-7a162ccee264 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:51.307783 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371b24a9-bc73-4b92-a4f9-7a162ccee264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:51.342086 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-371b24a9-bc73-4b92-a4f9-7a162ccee264 None None] MFA Rules not processed for user `ff7a95c0e54a40adb33ea6b152406554`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:34:51.343324 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-371b24a9-bc73-4b92-a4f9-7a162ccee264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:34:51.410435 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:34:51.421697 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1082/2151] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:34:51 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:34:51.436389 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26284b69-ed03-4855-8f5d-b99ec2dcda8b service nova] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:34:51.448355 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26284b69-ed03-4855-8f5d-b99ec2dcda8b service nova] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:34:51.454117 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fe1c9c-954a-489e-983a-dda449b17728 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87adc1d5bc240e99f95f6595b350be4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:34:51.454748 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fe1c9c-954a-489e-983a-dda449b17728 service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:51.455081 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fe1c9c-954a-489e-983a-dda449b17728 service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:51.455894 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fe1c9c-954a-489e-983a-dda449b17728 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:51.458097 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-b6fe1c9c-954a-489e-983a-dda449b17728 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:34:51.534794 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fe1c9c-954a-489e-983a-dda449b17728 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:34:51.536606 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fe1c9c-954a-489e-983a-dda449b17728 service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:34:51.553074 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1070/2152] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:34:51 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:34:52.841278 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3876869e-3e9f-443b-8ee4-09949f62561e None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:52.841759 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3876869e-3e9f-443b-8ee4-09949f62561e None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:52.841870 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3876869e-3e9f-443b-8ee4-09949f62561e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:52.866104 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-3876869e-3e9f-443b-8ee4-09949f62561e None None] MFA Rules not processed for user `cbb2fa7d4e5b42388cb4a97ec3495d51`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:34:52.867511 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-3876869e-3e9f-443b-8ee4-09949f62561e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:34:52.877040 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1083/2153] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:34:52 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:35:01.542248 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=44 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:35:02.859971 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=5 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:35:04.748527 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fe1c9c-954a-489e-983a-dda449b17728 service nova] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:04.821389 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fe1c9c-954a-489e-983a-dda449b17728 service nova] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:04.826282 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014e95c5-8049-4e18-95e3-3144e9ecb3af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:04.826942 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014e95c5-8049-4e18-95e3-3144e9ecb3af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:04.827218 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014e95c5-8049-4e18-95e3-3144e9ecb3af admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:04.827492 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014e95c5-8049-4e18-95e3-3144e9ecb3af admin admin] PATH_INFO: `/v3/users/2846129dffcd40abbd92eca26c271800` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:04.828729 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014e95c5-8049-4e18-95e3-3144e9ecb3af admin admin] RBAC: Authorizing `identity:delete_user(user_id=2846129dffcd40abbd92eca26c271800)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:04.830761 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014e95c5-8049-4e18-95e3-3144e9ecb3af admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:04.880421 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-014e95c5-8049-4e18-95e3-3144e9ecb3af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2846129dffcd40abbd92eca26c271800'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:04.880983 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-014e95c5-8049-4e18-95e3-3144e9ecb3af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2846129dffcd40abbd92eca26c271800'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:04.881620 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-014e95c5-8049-4e18-95e3-3144e9ecb3af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2846129dffcd40abbd92eca26c271800'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:04.893092 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-014e95c5-8049-4e18-95e3-3144e9ecb3af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2846129dffcd40abbd92eca26c271800'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:04.900678 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-014e95c5-8049-4e18-95e3-3144e9ecb3af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2846129dffcd40abbd92eca26c271800'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:04.916702 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1071/2154] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Aug 6 01:35:04 2022] DELETE /identity/v3/users/2846129dffcd40abbd92eca26c271800 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:35:04.919549 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7160190b-5bb4-4490-999d-047fd8ed5749 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:04.920173 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7160190b-5bb4-4490-999d-047fd8ed5749 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:04.920624 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7160190b-5bb4-4490-999d-047fd8ed5749 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:04.944482 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-7160190b-5bb4-4490-999d-047fd8ed5749 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:35:04.945743 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-7160190b-5bb4-4490-999d-047fd8ed5749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:35:05.007978 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1084/2155] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:35:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:35:05.017785 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014e95c5-8049-4e18-95e3-3144e9ecb3af admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:05.019792 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-014e95c5-8049-4e18-95e3-3144e9ecb3af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:35:05.079063 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014e95c5-8049-4e18-95e3-3144e9ecb3af admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:05.082861 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8882477a-6356-46e4-8729-857852ecc916 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:05.083482 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8882477a-6356-46e4-8729-857852ecc916 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:05.083766 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8882477a-6356-46e4-8729-857852ecc916 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:05.084044 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8882477a-6356-46e4-8729-857852ecc916 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:05.098614 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8882477a-6356-46e4-8729-857852ecc916 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:05.100150 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8882477a-6356-46e4-8729-857852ecc916 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:05.110185 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1072/2156] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:35:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:35:05.469787 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7160190b-5bb4-4490-999d-047fd8ed5749 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:05.471215 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-7160190b-5bb4-4490-999d-047fd8ed5749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:35:05.484927 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7160190b-5bb4-4490-999d-047fd8ed5749 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:05.491815 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b76fd7-9f51-4ee0-8f44-577fdc11fce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:05.492494 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b76fd7-9f51-4ee0-8f44-577fdc11fce5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:05.493000 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b76fd7-9f51-4ee0-8f44-577fdc11fce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:05.493352 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b76fd7-9f51-4ee0-8f44-577fdc11fce5 admin admin] PATH_INFO: `/v3/projects/61190280a86846b5afd9b0d768f89713` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:05.494740 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b76fd7-9f51-4ee0-8f44-577fdc11fce5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=61190280a86846b5afd9b0d768f89713)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:05.497032 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b76fd7-9f51-4ee0-8f44-577fdc11fce5 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:05.554388 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-87b76fd7-9f51-4ee0-8f44-577fdc11fce5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61190280a86846b5afd9b0d768f89713', 'request_id': 'req-87b76fd7-9f51-4ee0-8f44-577fdc11fce5'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:05.560534 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-87b76fd7-9f51-4ee0-8f44-577fdc11fce5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61190280a86846b5afd9b0d768f89713', 'request_id': 'req-87b76fd7-9f51-4ee0-8f44-577fdc11fce5'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:05.566499 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-87b76fd7-9f51-4ee0-8f44-577fdc11fce5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61190280a86846b5afd9b0d768f89713', 'request_id': 'req-87b76fd7-9f51-4ee0-8f44-577fdc11fce5'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:05.580204 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-87b76fd7-9f51-4ee0-8f44-577fdc11fce5 admin admin] The token cache is being invalidate because project 61190280a86846b5afd9b0d768f89713 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:35:05.580498 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-87b76fd7-9f51-4ee0-8f44-577fdc11fce5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:05.581373 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-87b76fd7-9f51-4ee0-8f44-577fdc11fce5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:05.582971 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1085/2157] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Aug 6 01:35:05 2022] DELETE /identity/v3/projects/61190280a86846b5afd9b0d768f89713 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:35:05.588021 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881f0049-bcc0-4beb-a1c2-84c2499926f0 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:05.588309 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881f0049-bcc0-4beb-a1c2-84c2499926f0 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:05.588585 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881f0049-bcc0-4beb-a1c2-84c2499926f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:05.608039 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-881f0049-bcc0-4beb-a1c2-84c2499926f0 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:35:05.609271 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-881f0049-bcc0-4beb-a1c2-84c2499926f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:35:05.628586 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:35:05.689021 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1073/2158] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:35:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:35:05.695946 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cdbe64-d9ab-4d55-8e04-bc7eb20357ce None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:05.696240 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cdbe64-d9ab-4d55-8e04-bc7eb20357ce None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:05.696524 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cdbe64-d9ab-4d55-8e04-bc7eb20357ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:05.717732 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-a5cdbe64-d9ab-4d55-8e04-bc7eb20357ce None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:35:05.718891 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-a5cdbe64-d9ab-4d55-8e04-bc7eb20357ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:35:05.727846 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1086/2159] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:35:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:35:05.732139 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881f0049-bcc0-4beb-a1c2-84c2499926f0 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:05.743655 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881f0049-bcc0-4beb-a1c2-84c2499926f0 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:05.748393 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ddb3b9-1203-4a9c-b310-6b0fb439e5a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:05.749025 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ddb3b9-1203-4a9c-b310-6b0fb439e5a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:05.749818 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ddb3b9-1203-4a9c-b310-6b0fb439e5a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:05.749818 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ddb3b9-1203-4a9c-b310-6b0fb439e5a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:05.751123 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ddb3b9-1203-4a9c-b310-6b0fb439e5a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:35:05.751123 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ddb3b9-1203-4a9c-b310-6b0fb439e5a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:05.753455 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ddb3b9-1203-4a9c-b310-6b0fb439e5a7 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:05.776316 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1074/2160] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Aug 6 01:35:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:35:05.780664 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cdbe64-d9ab-4d55-8e04-bc7eb20357ce None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:05.791596 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cdbe64-d9ab-4d55-8e04-bc7eb20357ce None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:05.796861 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2213e7ac-4ed5-4133-91b2-a885b79c578d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:05.797547 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2213e7ac-4ed5-4133-91b2-a885b79c578d admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:05.797829 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2213e7ac-4ed5-4133-91b2-a885b79c578d admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:05.798128 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2213e7ac-4ed5-4133-91b2-a885b79c578d admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:05.798938 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2213e7ac-4ed5-4133-91b2-a885b79c578d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:05.800839 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2213e7ac-4ed5-4133-91b2-a885b79c578d admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:05.834232 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1087/2161] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Aug 6 01:35:05 2022] POST /identity/v3/projects => generated 371 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:35:05.835247 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ddb3b9-1203-4a9c-b310-6b0fb439e5a7 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:05.846504 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ddb3b9-1203-4a9c-b310-6b0fb439e5a7 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:05.913687 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f021fd97-70b5-4536-a530-2153510419eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:05.914311 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f021fd97-70b5-4536-a530-2153510419eb admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:05.914588 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f021fd97-70b5-4536-a530-2153510419eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:05.914890 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f021fd97-70b5-4536-a530-2153510419eb admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:05.915589 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f021fd97-70b5-4536-a530-2153510419eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:05.917522 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f021fd97-70b5-4536-a530-2153510419eb admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:05.938286 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1075/2162] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:35:05 2022] POST /identity/v3/users => generated 286 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:35:05.942739 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2213e7ac-4ed5-4133-91b2-a885b79c578d admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:05.953588 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2213e7ac-4ed5-4133-91b2-a885b79c578d admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:05.959017 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42073ec2-0d2c-47db-af2d-ddb18f585243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:05.959895 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42073ec2-0d2c-47db-af2d-ddb18f585243 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:05.960484 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42073ec2-0d2c-47db-af2d-ddb18f585243 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:05.961091 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42073ec2-0d2c-47db-af2d-ddb18f585243 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:05.961985 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42073ec2-0d2c-47db-af2d-ddb18f585243 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:05.963615 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42073ec2-0d2c-47db-af2d-ddb18f585243 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:05.975259 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1088/2163] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:35:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:35:05.978965 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f021fd97-70b5-4536-a530-2153510419eb admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:05.988971 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f021fd97-70b5-4536-a530-2153510419eb admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:05.993941 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158ac3fc-73e5-408c-b2d5-62a849b793b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:05.994550 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158ac3fc-73e5-408c-b2d5-62a849b793b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:05.994834 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158ac3fc-73e5-408c-b2d5-62a849b793b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:05.995113 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158ac3fc-73e5-408c-b2d5-62a849b793b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:05.995834 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158ac3fc-73e5-408c-b2d5-62a849b793b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:05.997449 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158ac3fc-73e5-408c-b2d5-62a849b793b8 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:06.008607 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1076/2164] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:35:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:35:06.012570 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42073ec2-0d2c-47db-af2d-ddb18f585243 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:06.023843 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42073ec2-0d2c-47db-af2d-ddb18f585243 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:06.028598 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0323c3-d75e-41c4-aed3-ff6347c07f4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:06.029321 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0323c3-d75e-41c4-aed3-ff6347c07f4e admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:06.029603 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0323c3-d75e-41c4-aed3-ff6347c07f4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:06.029901 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0323c3-d75e-41c4-aed3-ff6347c07f4e admin admin] PATH_INFO: `/v3/projects/0ac6c86684404d59b781c4489c8cc5fe/users/3b05532eb4ff45a8861a07291298a212/roles/61b3b93d37a047e4bb8369b46fe20db9` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:06.044871 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0323c3-d75e-41c4-aed3-ff6347c07f4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ac6c86684404d59b781c4489c8cc5fe, user_id=3b05532eb4ff45a8861a07291298a212, role_id=61b3b93d37a047e4bb8369b46fe20db9)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:06.047554 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0323c3-d75e-41c4-aed3-ff6347c07f4e admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:06.063222 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1089/2165] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:35:06 2022] PUT /identity/v3/projects/0ac6c86684404d59b781c4489c8cc5fe/users/3b05532eb4ff45a8861a07291298a212/roles/61b3b93d37a047e4bb8369b46fe20db9 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:35:06.074560 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e198e601-e5e3-40d2-9c4a-c80715bcdb0e None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:06.074849 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e198e601-e5e3-40d2-9c4a-c80715bcdb0e None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:06.075135 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e198e601-e5e3-40d2-9c4a-c80715bcdb0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:06.103588 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-e198e601-e5e3-40d2-9c4a-c80715bcdb0e None None] MFA Rules not processed for user `3b05532eb4ff45a8861a07291298a212`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:35:06.104808 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-e198e601-e5e3-40d2-9c4a-c80715bcdb0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:35:06.175649 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1077/2166] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:35:06 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:35:06.186185 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0323c3-d75e-41c4-aed3-ff6347c07f4e admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:06.187910 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-3c0323c3-d75e-41c4-aed3-ff6347c07f4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:35:06.210374 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:35:06.269757 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0323c3-d75e-41c4-aed3-ff6347c07f4e admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:06.275973 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36664771-8daf-400d-b625-6f20305e5104 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87adc1d5bc240e99f95f6595b350be4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:06.276567 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36664771-8daf-400d-b625-6f20305e5104 service nova] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:06.276845 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36664771-8daf-400d-b625-6f20305e5104 service nova] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:06.277147 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36664771-8daf-400d-b625-6f20305e5104 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:06.284918 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36664771-8daf-400d-b625-6f20305e5104 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:06.286615 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36664771-8daf-400d-b625-6f20305e5104 service nova] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:06.295199 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1090/2167] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:35:06 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:35:10.054444 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e198e601-e5e3-40d2-9c4a-c80715bcdb0e None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:10.073653 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:35:10.117365 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e198e601-e5e3-40d2-9c4a-c80715bcdb0e None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:10.122419 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf4ade2-33bb-4471-aab3-c219e2019481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:10.123056 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf4ade2-33bb-4471-aab3-c219e2019481 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:10.123346 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf4ade2-33bb-4471-aab3-c219e2019481 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:10.123625 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf4ade2-33bb-4471-aab3-c219e2019481 admin admin] PATH_INFO: `/v3/users/3b05532eb4ff45a8861a07291298a212` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:10.124789 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf4ade2-33bb-4471-aab3-c219e2019481 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b05532eb4ff45a8861a07291298a212)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:10.126787 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf4ade2-33bb-4471-aab3-c219e2019481 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:10.173326 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-dbf4ade2-33bb-4471-aab3-c219e2019481 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b05532eb4ff45a8861a07291298a212'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:10.173981 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-dbf4ade2-33bb-4471-aab3-c219e2019481 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b05532eb4ff45a8861a07291298a212'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:10.174589 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-dbf4ade2-33bb-4471-aab3-c219e2019481 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b05532eb4ff45a8861a07291298a212'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:10.188266 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-dbf4ade2-33bb-4471-aab3-c219e2019481 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b05532eb4ff45a8861a07291298a212'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:10.195467 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-dbf4ade2-33bb-4471-aab3-c219e2019481 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b05532eb4ff45a8861a07291298a212'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:10.209097 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1078/2168] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Aug 6 01:35:10 2022] DELETE /identity/v3/users/3b05532eb4ff45a8861a07291298a212 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:35:10.213689 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209c3d84-acd9-4720-b57e-712672f469cb None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:10.214187 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209c3d84-acd9-4720-b57e-712672f469cb None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:10.214281 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209c3d84-acd9-4720-b57e-712672f469cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:10.232630 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-209c3d84-acd9-4720-b57e-712672f469cb None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:35:10.233860 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-209c3d84-acd9-4720-b57e-712672f469cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:35:10.293211 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1091/2169] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:35:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:35:10.302664 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf4ade2-33bb-4471-aab3-c219e2019481 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:10.304073 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-dbf4ade2-33bb-4471-aab3-c219e2019481 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:35:10.371721 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf4ade2-33bb-4471-aab3-c219e2019481 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:10.376182 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a63d4d5-23fd-41b3-8990-1c18d72e11d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:10.376984 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a63d4d5-23fd-41b3-8990-1c18d72e11d3 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:10.377451 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a63d4d5-23fd-41b3-8990-1c18d72e11d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:10.377884 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a63d4d5-23fd-41b3-8990-1c18d72e11d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:10.389414 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a63d4d5-23fd-41b3-8990-1c18d72e11d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:10.390886 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a63d4d5-23fd-41b3-8990-1c18d72e11d3 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:10.402198 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1079/2170] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:35:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:35:10.749501 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209c3d84-acd9-4720-b57e-712672f469cb None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:10.750970 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-209c3d84-acd9-4720-b57e-712672f469cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:35:10.766443 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209c3d84-acd9-4720-b57e-712672f469cb None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:10.771198 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd817b23-2386-4a0e-af2b-d2c1f6e15c9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:10.771842 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd817b23-2386-4a0e-af2b-d2c1f6e15c9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:10.772142 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd817b23-2386-4a0e-af2b-d2c1f6e15c9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:10.772763 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd817b23-2386-4a0e-af2b-d2c1f6e15c9d admin admin] PATH_INFO: `/v3/projects/0ac6c86684404d59b781c4489c8cc5fe` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:10.773785 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd817b23-2386-4a0e-af2b-d2c1f6e15c9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ac6c86684404d59b781c4489c8cc5fe)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:10.776297 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd817b23-2386-4a0e-af2b-d2c1f6e15c9d admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:10.827054 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-fd817b23-2386-4a0e-af2b-d2c1f6e15c9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ac6c86684404d59b781c4489c8cc5fe', 'request_id': 'req-fd817b23-2386-4a0e-af2b-d2c1f6e15c9d'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:10.833292 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-fd817b23-2386-4a0e-af2b-d2c1f6e15c9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ac6c86684404d59b781c4489c8cc5fe', 'request_id': 'req-fd817b23-2386-4a0e-af2b-d2c1f6e15c9d'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:10.839463 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-fd817b23-2386-4a0e-af2b-d2c1f6e15c9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ac6c86684404d59b781c4489c8cc5fe', 'request_id': 'req-fd817b23-2386-4a0e-af2b-d2c1f6e15c9d'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:10.852954 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-fd817b23-2386-4a0e-af2b-d2c1f6e15c9d admin admin] The token cache is being invalidate because project 0ac6c86684404d59b781c4489c8cc5fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:35:10.853252 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-fd817b23-2386-4a0e-af2b-d2c1f6e15c9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:10.853893 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-fd817b23-2386-4a0e-af2b-d2c1f6e15c9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:10.858895 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1092/2171] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Aug 6 01:35:10 2022] DELETE /identity/v3/projects/0ac6c86684404d59b781c4489c8cc5fe => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:35:10.860783 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b56017-73e4-46e7-970f-caed4e1b0db2 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:10.861077 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b56017-73e4-46e7-970f-caed4e1b0db2 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:10.861364 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b56017-73e4-46e7-970f-caed4e1b0db2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:10.885659 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-75b56017-73e4-46e7-970f-caed4e1b0db2 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:35:10.886915 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-75b56017-73e4-46e7-970f-caed4e1b0db2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:35:10.922990 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:35:10.954582 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1080/2172] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:35:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:35:10.963275 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef30332-4f60-4149-b3b5-0f3656d0c1fa None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:10.963667 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef30332-4f60-4149-b3b5-0f3656d0c1fa None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:10.964019 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef30332-4f60-4149-b3b5-0f3656d0c1fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:10.992887 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-aef30332-4f60-4149-b3b5-0f3656d0c1fa None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:35:10.994373 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-aef30332-4f60-4149-b3b5-0f3656d0c1fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:35:11.002943 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1093/2173] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:35:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:35:11.007172 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b56017-73e4-46e7-970f-caed4e1b0db2 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:11.017262 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b56017-73e4-46e7-970f-caed4e1b0db2 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:11.022032 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c5a8e1-cbf7-4b28-8dca-7e8aef5f05da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:11.022642 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c5a8e1-cbf7-4b28-8dca-7e8aef5f05da admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:11.022919 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c5a8e1-cbf7-4b28-8dca-7e8aef5f05da admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:11.023192 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c5a8e1-cbf7-4b28-8dca-7e8aef5f05da admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:11.024601 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c5a8e1-cbf7-4b28-8dca-7e8aef5f05da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:35:11.025079 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c5a8e1-cbf7-4b28-8dca-7e8aef5f05da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:11.026707 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c5a8e1-cbf7-4b28-8dca-7e8aef5f05da admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:11.038987 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1081/2174] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Aug 6 01:35:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:35:11.043931 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef30332-4f60-4149-b3b5-0f3656d0c1fa None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:11.053694 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef30332-4f60-4149-b3b5-0f3656d0c1fa None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:11.058929 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ad2a7e-a897-4ac9-af7d-b1646c54f50d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:11.059544 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ad2a7e-a897-4ac9-af7d-b1646c54f50d admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:11.059826 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ad2a7e-a897-4ac9-af7d-b1646c54f50d admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:11.060108 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ad2a7e-a897-4ac9-af7d-b1646c54f50d admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:11.060774 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ad2a7e-a897-4ac9-af7d-b1646c54f50d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:11.062716 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ad2a7e-a897-4ac9-af7d-b1646c54f50d admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:11.096125 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:35:11.106067 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1094/2175] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Aug 6 01:35:11 2022] POST /identity/v3/projects => generated 371 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:35:11.110757 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c5a8e1-cbf7-4b28-8dca-7e8aef5f05da admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:11.123368 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c5a8e1-cbf7-4b28-8dca-7e8aef5f05da admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:11.182084 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb70eeb1-d4c1-4ed2-9a21-35c7ea7eab30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:11.182703 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb70eeb1-d4c1-4ed2-9a21-35c7ea7eab30 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:11.182981 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb70eeb1-d4c1-4ed2-9a21-35c7ea7eab30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:11.183253 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb70eeb1-d4c1-4ed2-9a21-35c7ea7eab30 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:11.183908 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb70eeb1-d4c1-4ed2-9a21-35c7ea7eab30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:11.185835 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb70eeb1-d4c1-4ed2-9a21-35c7ea7eab30 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:11.206211 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1082/2176] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:35:11 2022] POST /identity/v3/users => generated 286 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:35:11.210315 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ad2a7e-a897-4ac9-af7d-b1646c54f50d admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:11.220939 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ad2a7e-a897-4ac9-af7d-b1646c54f50d admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:11.225967 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d350d9-dc41-4d62-9057-ccd88d0a87fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:11.226600 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d350d9-dc41-4d62-9057-ccd88d0a87fc admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:11.227429 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d350d9-dc41-4d62-9057-ccd88d0a87fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:11.227706 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d350d9-dc41-4d62-9057-ccd88d0a87fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:11.228393 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d350d9-dc41-4d62-9057-ccd88d0a87fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:11.230012 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d350d9-dc41-4d62-9057-ccd88d0a87fc admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:11.240003 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1095/2177] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:35:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:35:11.244872 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb70eeb1-d4c1-4ed2-9a21-35c7ea7eab30 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:11.255114 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb70eeb1-d4c1-4ed2-9a21-35c7ea7eab30 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:11.259818 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e4f60b-1f4d-40e0-9835-f44375f40617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:11.261097 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e4f60b-1f4d-40e0-9835-f44375f40617 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:11.261097 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e4f60b-1f4d-40e0-9835-f44375f40617 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:11.261097 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e4f60b-1f4d-40e0-9835-f44375f40617 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:11.261726 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e4f60b-1f4d-40e0-9835-f44375f40617 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:11.263338 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e4f60b-1f4d-40e0-9835-f44375f40617 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:11.276070 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1083/2178] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:35:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:35:11.280072 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d350d9-dc41-4d62-9057-ccd88d0a87fc admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:11.291059 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d350d9-dc41-4d62-9057-ccd88d0a87fc admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:11.296117 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad0fbfb-ae4c-44c0-b1bb-7187bef931f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:11.296853 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad0fbfb-ae4c-44c0-b1bb-7187bef931f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:11.297173 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad0fbfb-ae4c-44c0-b1bb-7187bef931f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:11.297470 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad0fbfb-ae4c-44c0-b1bb-7187bef931f5 admin admin] PATH_INFO: `/v3/projects/ae944284b5874866a0795596f3a35e4d/users/e635c0021f294907b12241132286f7ca/roles/61b3b93d37a047e4bb8369b46fe20db9` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:11.313857 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad0fbfb-ae4c-44c0-b1bb-7187bef931f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae944284b5874866a0795596f3a35e4d, user_id=e635c0021f294907b12241132286f7ca, role_id=61b3b93d37a047e4bb8369b46fe20db9)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:11.316480 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad0fbfb-ae4c-44c0-b1bb-7187bef931f5 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:11.328899 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1096/2179] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:35:11 2022] PUT /identity/v3/projects/ae944284b5874866a0795596f3a35e4d/users/e635c0021f294907b12241132286f7ca/roles/61b3b93d37a047e4bb8369b46fe20db9 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:35:11.340523 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b81825-b5d2-4ac9-9a8b-22c92a47061c None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:11.340817 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b81825-b5d2-4ac9-9a8b-22c92a47061c None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:11.341130 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b81825-b5d2-4ac9-9a8b-22c92a47061c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:11.369996 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-f6b81825-b5d2-4ac9-9a8b-22c92a47061c None None] MFA Rules not processed for user `e635c0021f294907b12241132286f7ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:35:11.371207 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-f6b81825-b5d2-4ac9-9a8b-22c92a47061c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:35:11.436208 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1084/2180] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:35:11 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:35:11.440787 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad0fbfb-ae4c-44c0-b1bb-7187bef931f5 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:11.506635 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad0fbfb-ae4c-44c0-b1bb-7187bef931f5 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:11.511678 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9a409a-7463-43e8-b63b-7d771813f119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:11.512298 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9a409a-7463-43e8-b63b-7d771813f119 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:11.512576 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9a409a-7463-43e8-b63b-7d771813f119 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:11.512847 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9a409a-7463-43e8-b63b-7d771813f119 admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:11.513558 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9a409a-7463-43e8-b63b-7d771813f119 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:11.515461 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9a409a-7463-43e8-b63b-7d771813f119 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:11.545681 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1097/2181] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Aug 6 01:35:11 2022] POST /identity/v3/projects => generated 373 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:35:11.549551 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b81825-b5d2-4ac9-9a8b-22c92a47061c None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:11.560536 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b81825-b5d2-4ac9-9a8b-22c92a47061c None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:11.596518 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:35:11.624899 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d463a3-3d9c-486a-b082-6145210866e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:11.625546 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d463a3-3d9c-486a-b082-6145210866e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:11.625827 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d463a3-3d9c-486a-b082-6145210866e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:11.626102 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d463a3-3d9c-486a-b082-6145210866e0 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:11.626777 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d463a3-3d9c-486a-b082-6145210866e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:11.628709 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d463a3-3d9c-486a-b082-6145210866e0 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:11.652665 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1085/2182] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:35:11 2022] POST /identity/v3/users => generated 293 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:35:11.656856 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9a409a-7463-43e8-b63b-7d771813f119 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:11.668359 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9a409a-7463-43e8-b63b-7d771813f119 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:11.673113 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232c7aa0-4f92-4773-8ba0-7bc85ad9718b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:11.673728 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232c7aa0-4f92-4773-8ba0-7bc85ad9718b admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:11.674007 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232c7aa0-4f92-4773-8ba0-7bc85ad9718b admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:11.674282 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232c7aa0-4f92-4773-8ba0-7bc85ad9718b admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:11.674970 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232c7aa0-4f92-4773-8ba0-7bc85ad9718b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:11.676560 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232c7aa0-4f92-4773-8ba0-7bc85ad9718b admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:11.688205 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1098/2183] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:35:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 01:35:11.692492 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d463a3-3d9c-486a-b082-6145210866e0 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:11.703673 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d463a3-3d9c-486a-b082-6145210866e0 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:11.708427 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd8534a-c127-4f73-87ba-5a788872fa57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:11.709136 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd8534a-c127-4f73-87ba-5a788872fa57 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:11.709428 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd8534a-c127-4f73-87ba-5a788872fa57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:11.709709 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd8534a-c127-4f73-87ba-5a788872fa57 admin admin] PATH_INFO: `/v3/projects/86994f72baa34010bf47932af67353f5/users/8a2844b9b69147aca7c88515dcbd1bba/roles/4559b9a0837249f895a13c70096f8d38` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:11.726040 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd8534a-c127-4f73-87ba-5a788872fa57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=86994f72baa34010bf47932af67353f5, user_id=8a2844b9b69147aca7c88515dcbd1bba, role_id=4559b9a0837249f895a13c70096f8d38)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:11.730472 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd8534a-c127-4f73-87ba-5a788872fa57 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:11.749388 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1086/2184] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:35:11 2022] PUT /identity/v3/projects/86994f72baa34010bf47932af67353f5/users/8a2844b9b69147aca7c88515dcbd1bba/roles/4559b9a0837249f895a13c70096f8d38 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:35:11.753155 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232c7aa0-4f92-4773-8ba0-7bc85ad9718b admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:11.769652 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232c7aa0-4f92-4773-8ba0-7bc85ad9718b admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:11.839325 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f40ee5-5ba5-4df3-9691-f20a7c1b07df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:11.840063 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f40ee5-5ba5-4df3-9691-f20a7c1b07df admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:11.840408 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f40ee5-5ba5-4df3-9691-f20a7c1b07df admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:11.840754 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f40ee5-5ba5-4df3-9691-f20a7c1b07df admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:11.841556 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f40ee5-5ba5-4df3-9691-f20a7c1b07df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:11.843230 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f40ee5-5ba5-4df3-9691-f20a7c1b07df admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:11.855015 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1099/2185] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:35:11 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:35:11.859169 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd8534a-c127-4f73-87ba-5a788872fa57 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:11.877200 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd8534a-c127-4f73-87ba-5a788872fa57 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:11.883732 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d043c4d1-9187-4f5e-a8fc-dd5d73da414f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:11.890839 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d043c4d1-9187-4f5e-a8fc-dd5d73da414f admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:11.891517 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d043c4d1-9187-4f5e-a8fc-dd5d73da414f admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:11.891517 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d043c4d1-9187-4f5e-a8fc-dd5d73da414f admin admin] PATH_INFO: `/v3/projects/86994f72baa34010bf47932af67353f5/users/8a2844b9b69147aca7c88515dcbd1bba/roles/4559b9a0837249f895a13c70096f8d38` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:11.893886 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d043c4d1-9187-4f5e-a8fc-dd5d73da414f admin admin] RBAC: Authorizing `identity:create_grant(project_id=86994f72baa34010bf47932af67353f5, user_id=8a2844b9b69147aca7c88515dcbd1bba, role_id=4559b9a0837249f895a13c70096f8d38)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:11.896618 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d043c4d1-9187-4f5e-a8fc-dd5d73da414f admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:11.909096 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1087/2186] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:35:11 2022] PUT /identity/v3/projects/86994f72baa34010bf47932af67353f5/users/8a2844b9b69147aca7c88515dcbd1bba/roles/4559b9a0837249f895a13c70096f8d38 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:35:11.920386 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92ce718-47ac-430c-888a-845e86517aaf None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:11.920679 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92ce718-47ac-430c-888a-845e86517aaf None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:11.920963 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92ce718-47ac-430c-888a-845e86517aaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:11.952545 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-c92ce718-47ac-430c-888a-845e86517aaf None None] MFA Rules not processed for user `8a2844b9b69147aca7c88515dcbd1bba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:35:11.953899 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-c92ce718-47ac-430c-888a-845e86517aaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:35:11.982208 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:35:12.034441 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1100/2187] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:35:11 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:35:12.046609 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d043c4d1-9187-4f5e-a8fc-dd5d73da414f admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:12.048432 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-d043c4d1-9187-4f5e-a8fc-dd5d73da414f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:35:12.114501 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d043c4d1-9187-4f5e-a8fc-dd5d73da414f admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:12.119243 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2284cb9-29e7-425e-815c-060737e323a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1be1e6173e374a60b9e1fbf51a1ab571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:12.119869 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2284cb9-29e7-425e-815c-060737e323a8 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:12.120152 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2284cb9-29e7-425e-815c-060737e323a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:12.120431 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2284cb9-29e7-425e-815c-060737e323a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:12.129626 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2284cb9-29e7-425e-815c-060737e323a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:12.131058 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2284cb9-29e7-425e-815c-060737e323a8 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:12.141757 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1088/2188] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:35:12 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:35:12.612977 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92ce718-47ac-430c-888a-845e86517aaf None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:12.682990 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92ce718-47ac-430c-888a-845e86517aaf None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:12.688029 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01de7edc-c2b5-4567-93bb-3aa8e4cc4245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:12.688683 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01de7edc-c2b5-4567-93bb-3aa8e4cc4245 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:12.688965 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01de7edc-c2b5-4567-93bb-3aa8e4cc4245 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:12.689311 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01de7edc-c2b5-4567-93bb-3aa8e4cc4245 admin admin] PATH_INFO: `/v3/users/e635c0021f294907b12241132286f7ca` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:12.690556 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01de7edc-c2b5-4567-93bb-3aa8e4cc4245 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e635c0021f294907b12241132286f7ca)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:12.692533 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01de7edc-c2b5-4567-93bb-3aa8e4cc4245 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:12.740045 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-01de7edc-c2b5-4567-93bb-3aa8e4cc4245 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e635c0021f294907b12241132286f7ca'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:12.747911 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-01de7edc-c2b5-4567-93bb-3aa8e4cc4245 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e635c0021f294907b12241132286f7ca'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:12.748584 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-01de7edc-c2b5-4567-93bb-3aa8e4cc4245 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e635c0021f294907b12241132286f7ca'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:12.749242 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-01de7edc-c2b5-4567-93bb-3aa8e4cc4245 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e635c0021f294907b12241132286f7ca'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:12.764196 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-01de7edc-c2b5-4567-93bb-3aa8e4cc4245 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e635c0021f294907b12241132286f7ca'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:12.777998 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1101/2189] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Aug 6 01:35:12 2022] DELETE /identity/v3/users/e635c0021f294907b12241132286f7ca => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:35:12.782511 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08f7ab3-b17d-4b30-a17b-dbf2baa9948b None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:12.782801 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08f7ab3-b17d-4b30-a17b-dbf2baa9948b None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:12.783075 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08f7ab3-b17d-4b30-a17b-dbf2baa9948b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:12.801512 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-a08f7ab3-b17d-4b30-a17b-dbf2baa9948b None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:35:12.802738 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-a08f7ab3-b17d-4b30-a17b-dbf2baa9948b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:35:12.875379 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1089/2190] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:35:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:35:12.885414 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01de7edc-c2b5-4567-93bb-3aa8e4cc4245 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:12.886845 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-01de7edc-c2b5-4567-93bb-3aa8e4cc4245 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:35:12.933503 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:35:12.949278 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01de7edc-c2b5-4567-93bb-3aa8e4cc4245 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:12.952879 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b411c0-0275-4ccd-bae9-6a904edae3c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:12.953521 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b411c0-0275-4ccd-bae9-6a904edae3c4 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:12.953799 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b411c0-0275-4ccd-bae9-6a904edae3c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:12.954067 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b411c0-0275-4ccd-bae9-6a904edae3c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:12.963613 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b411c0-0275-4ccd-bae9-6a904edae3c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:12.965029 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b411c0-0275-4ccd-bae9-6a904edae3c4 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:12.974377 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1102/2191] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:35:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:35:13.276305 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08f7ab3-b17d-4b30-a17b-dbf2baa9948b None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:13.278068 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-a08f7ab3-b17d-4b30-a17b-dbf2baa9948b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:35:13.308696 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08f7ab3-b17d-4b30-a17b-dbf2baa9948b None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:13.313460 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857bcdb6-f62a-4a2b-97db-767c1c8898d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:13.314103 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857bcdb6-f62a-4a2b-97db-767c1c8898d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:13.314384 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857bcdb6-f62a-4a2b-97db-767c1c8898d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:13.314661 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857bcdb6-f62a-4a2b-97db-767c1c8898d6 admin admin] PATH_INFO: `/v3/projects/ae944284b5874866a0795596f3a35e4d` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:13.315831 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857bcdb6-f62a-4a2b-97db-767c1c8898d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae944284b5874866a0795596f3a35e4d)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:13.317801 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857bcdb6-f62a-4a2b-97db-767c1c8898d6 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:13.348943 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:35:13.368847 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-857bcdb6-f62a-4a2b-97db-767c1c8898d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae944284b5874866a0795596f3a35e4d', 'request_id': 'req-857bcdb6-f62a-4a2b-97db-767c1c8898d6'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:13.374912 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-857bcdb6-f62a-4a2b-97db-767c1c8898d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae944284b5874866a0795596f3a35e4d', 'request_id': 'req-857bcdb6-f62a-4a2b-97db-767c1c8898d6'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:13.380558 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-857bcdb6-f62a-4a2b-97db-767c1c8898d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae944284b5874866a0795596f3a35e4d', 'request_id': 'req-857bcdb6-f62a-4a2b-97db-767c1c8898d6'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:13.395184 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-857bcdb6-f62a-4a2b-97db-767c1c8898d6 admin admin] The token cache is being invalidate because project ae944284b5874866a0795596f3a35e4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:35:13.395474 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-857bcdb6-f62a-4a2b-97db-767c1c8898d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:13.396104 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-857bcdb6-f62a-4a2b-97db-767c1c8898d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:13.397667 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1090/2192] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Aug 6 01:35:13 2022] DELETE /identity/v3/projects/ae944284b5874866a0795596f3a35e4d => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:35:13.402780 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b411c0-0275-4ccd-bae9-6a904edae3c4 service neutron] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:13.412767 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b411c0-0275-4ccd-bae9-6a904edae3c4 service neutron] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:13.471232 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2372c22b-17b1-4345-befd-c7c6a089d127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:13.471902 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2372c22b-17b1-4345-befd-c7c6a089d127 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:13.472185 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2372c22b-17b1-4345-befd-c7c6a089d127 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:13.472463 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2372c22b-17b1-4345-befd-c7c6a089d127 admin admin] PATH_INFO: `/v3/users/8a2844b9b69147aca7c88515dcbd1bba` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:13.473675 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2372c22b-17b1-4345-befd-c7c6a089d127 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a2844b9b69147aca7c88515dcbd1bba)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:13.475587 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2372c22b-17b1-4345-befd-c7c6a089d127 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:13.529005 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-2372c22b-17b1-4345-befd-c7c6a089d127 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a2844b9b69147aca7c88515dcbd1bba'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:13.536759 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-2372c22b-17b1-4345-befd-c7c6a089d127 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a2844b9b69147aca7c88515dcbd1bba'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:13.537498 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-2372c22b-17b1-4345-befd-c7c6a089d127 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a2844b9b69147aca7c88515dcbd1bba'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:13.538114 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-2372c22b-17b1-4345-befd-c7c6a089d127 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a2844b9b69147aca7c88515dcbd1bba'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:13.551612 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-2372c22b-17b1-4345-befd-c7c6a089d127 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a2844b9b69147aca7c88515dcbd1bba'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:13.568152 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1103/2193] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Aug 6 01:35:13 2022] DELETE /identity/v3/users/8a2844b9b69147aca7c88515dcbd1bba => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:35:13.850744 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857bcdb6-f62a-4a2b-97db-767c1c8898d6 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:13.852857 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-857bcdb6-f62a-4a2b-97db-767c1c8898d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:35:13.868988 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857bcdb6-f62a-4a2b-97db-767c1c8898d6 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:13.932729 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87efee71-1659-4124-80da-68444eecfddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:13.933136 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87efee71-1659-4124-80da-68444eecfddb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:13.933425 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87efee71-1659-4124-80da-68444eecfddb admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:13.933708 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87efee71-1659-4124-80da-68444eecfddb admin admin] PATH_INFO: `/v3/projects/86994f72baa34010bf47932af67353f5` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:13.934883 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87efee71-1659-4124-80da-68444eecfddb admin admin] RBAC: Authorizing `identity:delete_project(project_id=86994f72baa34010bf47932af67353f5)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:13.936794 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87efee71-1659-4124-80da-68444eecfddb admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:13.995433 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-87efee71-1659-4124-80da-68444eecfddb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86994f72baa34010bf47932af67353f5', 'request_id': 'req-87efee71-1659-4124-80da-68444eecfddb'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:14.001616 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-87efee71-1659-4124-80da-68444eecfddb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86994f72baa34010bf47932af67353f5', 'request_id': 'req-87efee71-1659-4124-80da-68444eecfddb'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:14.007309 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-87efee71-1659-4124-80da-68444eecfddb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86994f72baa34010bf47932af67353f5', 'request_id': 'req-87efee71-1659-4124-80da-68444eecfddb'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:14.019848 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-87efee71-1659-4124-80da-68444eecfddb admin admin] The token cache is being invalidate because project 86994f72baa34010bf47932af67353f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:35:14.020137 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-87efee71-1659-4124-80da-68444eecfddb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:14.020755 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-87efee71-1659-4124-80da-68444eecfddb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:14.022287 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1091/2194] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Aug 6 01:35:13 2022] DELETE /identity/v3/projects/86994f72baa34010bf47932af67353f5 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:35:14.027707 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac54b1f5-cd25-48ff-9a78-e19dc28973fb None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:14.028164 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac54b1f5-cd25-48ff-9a78-e19dc28973fb None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:14.028321 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac54b1f5-cd25-48ff-9a78-e19dc28973fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:14.048499 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-ac54b1f5-cd25-48ff-9a78-e19dc28973fb None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:35:14.049748 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-ac54b1f5-cd25-48ff-9a78-e19dc28973fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:35:14.108146 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1104/2195] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:35:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:35:14.117327 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809bf3bd-b034-42ec-8672-428003ecb16f None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:14.117624 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809bf3bd-b034-42ec-8672-428003ecb16f None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:14.117897 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809bf3bd-b034-42ec-8672-428003ecb16f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:14.137942 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-809bf3bd-b034-42ec-8672-428003ecb16f None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:35:14.139147 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-809bf3bd-b034-42ec-8672-428003ecb16f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:35:14.147726 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1092/2196] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:35:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:35:14.152789 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac54b1f5-cd25-48ff-9a78-e19dc28973fb None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:14.158671 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:35:14.164615 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac54b1f5-cd25-48ff-9a78-e19dc28973fb None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:14.170054 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9132957-9123-4898-9f3f-57e3c928a6ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:14.170671 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9132957-9123-4898-9f3f-57e3c928a6ff admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:14.170952 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9132957-9123-4898-9f3f-57e3c928a6ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:14.171228 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9132957-9123-4898-9f3f-57e3c928a6ff admin admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:14.171927 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9132957-9123-4898-9f3f-57e3c928a6ff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:35:14.172232 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9132957-9123-4898-9f3f-57e3c928a6ff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:14.173867 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9132957-9123-4898-9f3f-57e3c928a6ff admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:14.185352 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1105/2197] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Aug 6 01:35:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:35:14.189867 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809bf3bd-b034-42ec-8672-428003ecb16f None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:14.199893 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809bf3bd-b034-42ec-8672-428003ecb16f None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:14.204585 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1322223-200a-4fdb-bd6f-571e6cb65383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:14.205234 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1322223-200a-4fdb-bd6f-571e6cb65383 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:14.205519 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1322223-200a-4fdb-bd6f-571e6cb65383 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:14.205796 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1322223-200a-4fdb-bd6f-571e6cb65383 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:14.206470 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1322223-200a-4fdb-bd6f-571e6cb65383 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:14.208398 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1322223-200a-4fdb-bd6f-571e6cb65383 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:14.235152 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1093/2198] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Aug 6 01:35:14 2022] POST /identity/v3/projects => generated 365 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:35:14.239293 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9132957-9123-4898-9f3f-57e3c928a6ff admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:14.250405 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9132957-9123-4898-9f3f-57e3c928a6ff admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:14.305957 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e50d49-f6b4-4677-a068-27c52bca983d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:14.306797 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e50d49-f6b4-4677-a068-27c52bca983d admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:14.307087 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e50d49-f6b4-4677-a068-27c52bca983d admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:14.307363 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e50d49-f6b4-4677-a068-27c52bca983d admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:14.308098 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e50d49-f6b4-4677-a068-27c52bca983d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:14.310193 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e50d49-f6b4-4677-a068-27c52bca983d admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:14.334395 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1106/2199] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:35:14 2022] POST /identity/v3/users => generated 283 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:35:14.339326 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1322223-200a-4fdb-bd6f-571e6cb65383 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:14.349588 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1322223-200a-4fdb-bd6f-571e6cb65383 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:14.354879 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580d8206-1199-4ad5-babc-7d971e19e16a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:14.355491 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580d8206-1199-4ad5-babc-7d971e19e16a admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:14.355772 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580d8206-1199-4ad5-babc-7d971e19e16a admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:14.356049 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580d8206-1199-4ad5-babc-7d971e19e16a admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:14.356739 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580d8206-1199-4ad5-babc-7d971e19e16a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:14.358496 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580d8206-1199-4ad5-babc-7d971e19e16a admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:14.372182 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1094/2200] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:35:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:35:14.373382 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e50d49-f6b4-4677-a068-27c52bca983d admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:14.383884 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e50d49-f6b4-4677-a068-27c52bca983d admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:14.388705 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f929a38d-b773-4913-9ad1-3657aa64a8ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:14.389365 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f929a38d-b773-4913-9ad1-3657aa64a8ca admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:14.389654 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f929a38d-b773-4913-9ad1-3657aa64a8ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:14.389959 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f929a38d-b773-4913-9ad1-3657aa64a8ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:14.390663 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f929a38d-b773-4913-9ad1-3657aa64a8ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:14.392431 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f929a38d-b773-4913-9ad1-3657aa64a8ca admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:14.405314 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1107/2201] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:35:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:35:14.409294 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580d8206-1199-4ad5-babc-7d971e19e16a admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:14.417185 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:35:14.422167 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580d8206-1199-4ad5-babc-7d971e19e16a admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:14.430042 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86c21f2-816f-450f-8feb-25a2b248d6df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:14.430846 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86c21f2-816f-450f-8feb-25a2b248d6df admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:14.431129 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86c21f2-816f-450f-8feb-25a2b248d6df admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:14.431432 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86c21f2-816f-450f-8feb-25a2b248d6df admin admin] PATH_INFO: `/v3/projects/2ae7899beca9491d8b9af179a0b3cc2d/users/54369be5e11c4f949998cf9ab4600299/roles/61b3b93d37a047e4bb8369b46fe20db9` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:14.447130 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86c21f2-816f-450f-8feb-25a2b248d6df admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ae7899beca9491d8b9af179a0b3cc2d, user_id=54369be5e11c4f949998cf9ab4600299, role_id=61b3b93d37a047e4bb8369b46fe20db9)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:14.449832 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86c21f2-816f-450f-8feb-25a2b248d6df admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:14.463007 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1095/2202] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:35:14 2022] PUT /identity/v3/projects/2ae7899beca9491d8b9af179a0b3cc2d/users/54369be5e11c4f949998cf9ab4600299/roles/61b3b93d37a047e4bb8369b46fe20db9 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:35:14.473561 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d10ef1-f421-4d3e-80db-8247457a04eb None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:14.473883 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d10ef1-f421-4d3e-80db-8247457a04eb None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:14.474143 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d10ef1-f421-4d3e-80db-8247457a04eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:14.502058 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-71d10ef1-f421-4d3e-80db-8247457a04eb None None] MFA Rules not processed for user `54369be5e11c4f949998cf9ab4600299`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:35:14.503270 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-71d10ef1-f421-4d3e-80db-8247457a04eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:35:14.569040 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1108/2203] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:35:14 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:35:14.573751 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86c21f2-816f-450f-8feb-25a2b248d6df admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:14.638868 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86c21f2-816f-450f-8feb-25a2b248d6df admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:14.644689 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4e0753-6119-4341-a12a-dbb9ae2097f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:14.645588 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4e0753-6119-4341-a12a-dbb9ae2097f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:14.646010 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4e0753-6119-4341-a12a-dbb9ae2097f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:14.646418 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4e0753-6119-4341-a12a-dbb9ae2097f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:14.647209 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4e0753-6119-4341-a12a-dbb9ae2097f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:14.649666 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4e0753-6119-4341-a12a-dbb9ae2097f2 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:14.680610 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1096/2204] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Aug 6 01:35:14 2022] POST /identity/v3/projects => generated 365 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:35:14.686868 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d10ef1-f421-4d3e-80db-8247457a04eb None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:14.698362 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d10ef1-f421-4d3e-80db-8247457a04eb None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:14.751379 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:35:14.758728 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8472a9e3-f59e-4004-9a1f-26bfd6500b56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:14.759676 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8472a9e3-f59e-4004-9a1f-26bfd6500b56 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:14.759958 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8472a9e3-f59e-4004-9a1f-26bfd6500b56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:14.760233 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8472a9e3-f59e-4004-9a1f-26bfd6500b56 admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:14.760904 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8472a9e3-f59e-4004-9a1f-26bfd6500b56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:14.762870 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8472a9e3-f59e-4004-9a1f-26bfd6500b56 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:14.787824 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1109/2205] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:35:14 2022] POST /identity/v3/users => generated 289 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:35:14.789074 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4e0753-6119-4341-a12a-dbb9ae2097f2 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:14.807576 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4e0753-6119-4341-a12a-dbb9ae2097f2 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:14.813235 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0eabbc-70c0-4ea7-b5a8-91eb201c8ec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:14.813856 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0eabbc-70c0-4ea7-b5a8-91eb201c8ec5 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:14.814134 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0eabbc-70c0-4ea7-b5a8-91eb201c8ec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:14.814407 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0eabbc-70c0-4ea7-b5a8-91eb201c8ec5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:14.815105 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0eabbc-70c0-4ea7-b5a8-91eb201c8ec5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:14.816688 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0eabbc-70c0-4ea7-b5a8-91eb201c8ec5 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:14.828404 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1097/2206] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:35:14 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:35:14.832586 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8472a9e3-f59e-4004-9a1f-26bfd6500b56 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:14.843347 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8472a9e3-f59e-4004-9a1f-26bfd6500b56 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:14.848024 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd45983-e698-488d-9d39-b9f2d75ce720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:14.848684 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd45983-e698-488d-9d39-b9f2d75ce720 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:14.848960 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd45983-e698-488d-9d39-b9f2d75ce720 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:14.849278 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd45983-e698-488d-9d39-b9f2d75ce720 admin admin] PATH_INFO: `/v3/projects/989606ae4bee432284271ce0bb72cc84/users/47088cf70870459c8392080d036bb08d/roles/4559b9a0837249f895a13c70096f8d38` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:14.865953 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd45983-e698-488d-9d39-b9f2d75ce720 admin admin] RBAC: Authorizing `identity:create_grant(project_id=989606ae4bee432284271ce0bb72cc84, user_id=47088cf70870459c8392080d036bb08d, role_id=4559b9a0837249f895a13c70096f8d38)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:14.869476 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd45983-e698-488d-9d39-b9f2d75ce720 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:14.882027 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1110/2207] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:35:14 2022] PUT /identity/v3/projects/989606ae4bee432284271ce0bb72cc84/users/47088cf70870459c8392080d036bb08d/roles/4559b9a0837249f895a13c70096f8d38 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:35:14.887017 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0eabbc-70c0-4ea7-b5a8-91eb201c8ec5 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:14.903585 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0eabbc-70c0-4ea7-b5a8-91eb201c8ec5 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:14.965433 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e704572-7a35-4d1f-bb95-e1ddd2d92042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:14.966071 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e704572-7a35-4d1f-bb95-e1ddd2d92042 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:14.966411 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e704572-7a35-4d1f-bb95-e1ddd2d92042 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:14.966698 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e704572-7a35-4d1f-bb95-e1ddd2d92042 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:14.967396 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e704572-7a35-4d1f-bb95-e1ddd2d92042 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:14.969287 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e704572-7a35-4d1f-bb95-e1ddd2d92042 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:14.981945 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1098/2208] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:35:14 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 01:35:14.986248 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd45983-e698-488d-9d39-b9f2d75ce720 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:14.997987 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd45983-e698-488d-9d39-b9f2d75ce720 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:15.002788 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d6119f-fcd0-4fb4-ba1d-fb0293249d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:15.003519 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d6119f-fcd0-4fb4-ba1d-fb0293249d16 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:15.003802 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d6119f-fcd0-4fb4-ba1d-fb0293249d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:15.004076 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d6119f-fcd0-4fb4-ba1d-fb0293249d16 admin admin] PATH_INFO: `/v3/projects/989606ae4bee432284271ce0bb72cc84/users/47088cf70870459c8392080d036bb08d/roles/4559b9a0837249f895a13c70096f8d38` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:15.006066 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d6119f-fcd0-4fb4-ba1d-fb0293249d16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=989606ae4bee432284271ce0bb72cc84, user_id=47088cf70870459c8392080d036bb08d, role_id=4559b9a0837249f895a13c70096f8d38)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:15.008835 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d6119f-fcd0-4fb4-ba1d-fb0293249d16 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:15.019505 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1111/2209] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:35:14 2022] PUT /identity/v3/projects/989606ae4bee432284271ce0bb72cc84/users/47088cf70870459c8392080d036bb08d/roles/4559b9a0837249f895a13c70096f8d38 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:35:15.030052 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c3f978-5bb4-4dda-829b-7819d63f795d None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:15.030351 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c3f978-5bb4-4dda-829b-7819d63f795d None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:15.030639 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c3f978-5bb4-4dda-829b-7819d63f795d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:15.061746 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-19c3f978-5bb4-4dda-829b-7819d63f795d None None] MFA Rules not processed for user `47088cf70870459c8392080d036bb08d`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:35:15.062993 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-19c3f978-5bb4-4dda-829b-7819d63f795d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:35:15.102541 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:35:15.132573 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1099/2210] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:35:15 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:35:15.143827 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d6119f-fcd0-4fb4-ba1d-fb0293249d16 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:15.145698 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-a2d6119f-fcd0-4fb4-ba1d-fb0293249d16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:35:15.207309 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d6119f-fcd0-4fb4-ba1d-fb0293249d16 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:15.210933 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f13c00-468f-4c7c-bfcb-19bb8cb1601c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1be1e6173e374a60b9e1fbf51a1ab571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:15.211560 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f13c00-468f-4c7c-bfcb-19bb8cb1601c service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:15.211843 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f13c00-468f-4c7c-bfcb-19bb8cb1601c service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:15.212118 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f13c00-468f-4c7c-bfcb-19bb8cb1601c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:15.221136 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f13c00-468f-4c7c-bfcb-19bb8cb1601c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:15.222560 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f13c00-468f-4c7c-bfcb-19bb8cb1601c service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:15.233368 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1112/2211] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:35:15 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:35:15.646934 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c3f978-5bb4-4dda-829b-7819d63f795d None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:15.719132 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c3f978-5bb4-4dda-829b-7819d63f795d None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:15.723586 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7b4799-ac4e-4d45-b44b-d322bbda6970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:15.724234 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7b4799-ac4e-4d45-b44b-d322bbda6970 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:15.724498 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7b4799-ac4e-4d45-b44b-d322bbda6970 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:15.724776 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7b4799-ac4e-4d45-b44b-d322bbda6970 admin admin] PATH_INFO: `/v3/users/54369be5e11c4f949998cf9ab4600299` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:15.726001 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7b4799-ac4e-4d45-b44b-d322bbda6970 admin admin] RBAC: Authorizing `identity:delete_user(user_id=54369be5e11c4f949998cf9ab4600299)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:15.728153 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7b4799-ac4e-4d45-b44b-d322bbda6970 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:15.779705 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-2f7b4799-ac4e-4d45-b44b-d322bbda6970 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54369be5e11c4f949998cf9ab4600299'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:15.780453 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-2f7b4799-ac4e-4d45-b44b-d322bbda6970 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54369be5e11c4f949998cf9ab4600299'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:15.781093 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-2f7b4799-ac4e-4d45-b44b-d322bbda6970 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54369be5e11c4f949998cf9ab4600299'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:15.795138 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-2f7b4799-ac4e-4d45-b44b-d322bbda6970 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54369be5e11c4f949998cf9ab4600299'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:15.802015 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-2f7b4799-ac4e-4d45-b44b-d322bbda6970 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54369be5e11c4f949998cf9ab4600299'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:15.815465 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1100/2212] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Aug 6 01:35:15 2022] DELETE /identity/v3/users/54369be5e11c4f949998cf9ab4600299 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:35:15.820142 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09abbe04-adf1-41ad-86e7-7a26f57e23aa None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:15.820432 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09abbe04-adf1-41ad-86e7-7a26f57e23aa None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:15.821552 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09abbe04-adf1-41ad-86e7-7a26f57e23aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:15.840189 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-09abbe04-adf1-41ad-86e7-7a26f57e23aa None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:35:15.842205 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-09abbe04-adf1-41ad-86e7-7a26f57e23aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:35:15.922016 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1113/2213] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:35:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:35:15.927641 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7b4799-ac4e-4d45-b44b-d322bbda6970 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:15.929133 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-2f7b4799-ac4e-4d45-b44b-d322bbda6970 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:35:16.000891 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7b4799-ac4e-4d45-b44b-d322bbda6970 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:16.004741 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa36644-e2b5-4c6d-9488-37c81113048e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:16.005482 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa36644-e2b5-4c6d-9488-37c81113048e service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:16.005844 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa36644-e2b5-4c6d-9488-37c81113048e service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:16.006199 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa36644-e2b5-4c6d-9488-37c81113048e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:16.019946 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa36644-e2b5-4c6d-9488-37c81113048e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:16.019946 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa36644-e2b5-4c6d-9488-37c81113048e service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:16.024919 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:35:16.033087 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1101/2214] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:35:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:35:16.346423 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09abbe04-adf1-41ad-86e7-7a26f57e23aa None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:16.350098 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-09abbe04-adf1-41ad-86e7-7a26f57e23aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:35:16.378531 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09abbe04-adf1-41ad-86e7-7a26f57e23aa None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:16.387305 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3919e52-a853-4c01-9c73-28a76d0365c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:16.388104 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3919e52-a853-4c01-9c73-28a76d0365c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:16.388746 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3919e52-a853-4c01-9c73-28a76d0365c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:16.389129 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3919e52-a853-4c01-9c73-28a76d0365c7 admin admin] PATH_INFO: `/v3/projects/2ae7899beca9491d8b9af179a0b3cc2d` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:16.390302 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3919e52-a853-4c01-9c73-28a76d0365c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ae7899beca9491d8b9af179a0b3cc2d)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:16.392170 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3919e52-a853-4c01-9c73-28a76d0365c7 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:16.455012 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:35:16.456547 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-c3919e52-a853-4c01-9c73-28a76d0365c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ae7899beca9491d8b9af179a0b3cc2d', 'request_id': 'req-c3919e52-a853-4c01-9c73-28a76d0365c7'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:16.466247 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-c3919e52-a853-4c01-9c73-28a76d0365c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ae7899beca9491d8b9af179a0b3cc2d', 'request_id': 'req-c3919e52-a853-4c01-9c73-28a76d0365c7'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:16.472959 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-c3919e52-a853-4c01-9c73-28a76d0365c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ae7899beca9491d8b9af179a0b3cc2d', 'request_id': 'req-c3919e52-a853-4c01-9c73-28a76d0365c7'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:16.486461 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-c3919e52-a853-4c01-9c73-28a76d0365c7 admin admin] The token cache is being invalidate because project 2ae7899beca9491d8b9af179a0b3cc2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:35:16.486690 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-c3919e52-a853-4c01-9c73-28a76d0365c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:16.487309 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-c3919e52-a853-4c01-9c73-28a76d0365c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:16.488940 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1114/2215] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Aug 6 01:35:16 2022] DELETE /identity/v3/projects/2ae7899beca9491d8b9af179a0b3cc2d => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:35:16.493327 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa36644-e2b5-4c6d-9488-37c81113048e service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:16.503053 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa36644-e2b5-4c6d-9488-37c81113048e service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:16.560705 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be97fec1-31d1-40d6-ad3d-ddd75411446e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:16.561377 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be97fec1-31d1-40d6-ad3d-ddd75411446e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:16.561660 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be97fec1-31d1-40d6-ad3d-ddd75411446e admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:16.561934 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be97fec1-31d1-40d6-ad3d-ddd75411446e admin admin] PATH_INFO: `/v3/users/47088cf70870459c8392080d036bb08d` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:16.563080 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be97fec1-31d1-40d6-ad3d-ddd75411446e admin admin] RBAC: Authorizing `identity:delete_user(user_id=47088cf70870459c8392080d036bb08d)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:16.564979 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be97fec1-31d1-40d6-ad3d-ddd75411446e admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:16.619039 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-be97fec1-31d1-40d6-ad3d-ddd75411446e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47088cf70870459c8392080d036bb08d'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:16.619749 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-be97fec1-31d1-40d6-ad3d-ddd75411446e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47088cf70870459c8392080d036bb08d'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:16.620379 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-be97fec1-31d1-40d6-ad3d-ddd75411446e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47088cf70870459c8392080d036bb08d'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:16.632228 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-be97fec1-31d1-40d6-ad3d-ddd75411446e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47088cf70870459c8392080d036bb08d'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:16.641429 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-be97fec1-31d1-40d6-ad3d-ddd75411446e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47088cf70870459c8392080d036bb08d'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:16.656128 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1102/2216] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Aug 6 01:35:16 2022] DELETE /identity/v3/users/47088cf70870459c8392080d036bb08d => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:35:16.967284 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3919e52-a853-4c01-9c73-28a76d0365c7 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:16.968990 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-c3919e52-a853-4c01-9c73-28a76d0365c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:35:16.984854 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3919e52-a853-4c01-9c73-28a76d0365c7 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:17.049510 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0638744b-9305-4b4e-a1cb-81933151e508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:17.050374 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0638744b-9305-4b4e-a1cb-81933151e508 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:17.050756 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0638744b-9305-4b4e-a1cb-81933151e508 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:17.051126 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0638744b-9305-4b4e-a1cb-81933151e508 admin admin] PATH_INFO: `/v3/projects/989606ae4bee432284271ce0bb72cc84` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:17.052314 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0638744b-9305-4b4e-a1cb-81933151e508 admin admin] RBAC: Authorizing `identity:delete_project(project_id=989606ae4bee432284271ce0bb72cc84)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:17.054261 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0638744b-9305-4b4e-a1cb-81933151e508 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:17.109566 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-0638744b-9305-4b4e-a1cb-81933151e508 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '989606ae4bee432284271ce0bb72cc84', 'request_id': 'req-0638744b-9305-4b4e-a1cb-81933151e508'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:17.116301 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-0638744b-9305-4b4e-a1cb-81933151e508 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '989606ae4bee432284271ce0bb72cc84', 'request_id': 'req-0638744b-9305-4b4e-a1cb-81933151e508'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:17.122070 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-0638744b-9305-4b4e-a1cb-81933151e508 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '989606ae4bee432284271ce0bb72cc84', 'request_id': 'req-0638744b-9305-4b4e-a1cb-81933151e508'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:17.142578 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-0638744b-9305-4b4e-a1cb-81933151e508 admin admin] The token cache is being invalidate because project 989606ae4bee432284271ce0bb72cc84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:35:17.142963 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-0638744b-9305-4b4e-a1cb-81933151e508 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:17.143704 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-0638744b-9305-4b4e-a1cb-81933151e508 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:17.146151 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1115/2217] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Aug 6 01:35:16 2022] DELETE /identity/v3/projects/989606ae4bee432284271ce0bb72cc84 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:35:17.152203 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8e6b47-df67-4c93-a65d-0ef6ffac3989 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:17.152446 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8e6b47-df67-4c93-a65d-0ef6ffac3989 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:17.152899 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8e6b47-df67-4c93-a65d-0ef6ffac3989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:17.175972 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-cc8e6b47-df67-4c93-a65d-0ef6ffac3989 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:35:17.178695 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-cc8e6b47-df67-4c93-a65d-0ef6ffac3989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:35:17.241510 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1103/2218] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:35:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:35:17.250963 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8308392-dc8a-4bb2-a9a5-e22f6a8301bc None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:17.251421 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8308392-dc8a-4bb2-a9a5-e22f6a8301bc None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:17.251709 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8308392-dc8a-4bb2-a9a5-e22f6a8301bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:17.273146 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-f8308392-dc8a-4bb2-a9a5-e22f6a8301bc None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:35:17.274438 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-f8308392-dc8a-4bb2-a9a5-e22f6a8301bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:35:17.282422 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1116/2219] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:35:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:35:17.286730 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8e6b47-df67-4c93-a65d-0ef6ffac3989 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:17.296763 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8e6b47-df67-4c93-a65d-0ef6ffac3989 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:17.301955 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982b9e9d-86ff-4339-aa8c-68d33dfd8057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:17.302567 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982b9e9d-86ff-4339-aa8c-68d33dfd8057 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:17.302854 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982b9e9d-86ff-4339-aa8c-68d33dfd8057 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:17.303131 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982b9e9d-86ff-4339-aa8c-68d33dfd8057 admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:17.303836 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982b9e9d-86ff-4339-aa8c-68d33dfd8057 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:35:17.304148 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982b9e9d-86ff-4339-aa8c-68d33dfd8057 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:17.306433 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982b9e9d-86ff-4339-aa8c-68d33dfd8057 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:17.319012 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1104/2220] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Aug 6 01:35:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:35:17.324170 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8308392-dc8a-4bb2-a9a5-e22f6a8301bc None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:17.333728 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8308392-dc8a-4bb2-a9a5-e22f6a8301bc None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:17.339210 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57622ea-72af-47f9-8218-9aee1791762c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:17.339820 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57622ea-72af-47f9-8218-9aee1791762c admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:17.340102 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57622ea-72af-47f9-8218-9aee1791762c admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:17.340390 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57622ea-72af-47f9-8218-9aee1791762c admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:17.341059 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57622ea-72af-47f9-8218-9aee1791762c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:17.342978 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57622ea-72af-47f9-8218-9aee1791762c admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:17.375636 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1117/2221] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Aug 6 01:35:17 2022] POST /identity/v3/projects => generated 373 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:35:17.379819 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982b9e9d-86ff-4339-aa8c-68d33dfd8057 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:17.386849 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:35:17.391188 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982b9e9d-86ff-4339-aa8c-68d33dfd8057 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:17.472713 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc9fb38-38cd-411d-913c-ab449a58b36e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:17.473390 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc9fb38-38cd-411d-913c-ab449a58b36e admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:17.473674 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc9fb38-38cd-411d-913c-ab449a58b36e admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:17.473950 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc9fb38-38cd-411d-913c-ab449a58b36e admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:17.474625 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc9fb38-38cd-411d-913c-ab449a58b36e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:17.476530 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc9fb38-38cd-411d-913c-ab449a58b36e admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:17.497324 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1105/2222] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:35:17 2022] POST /identity/v3/users => generated 287 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:35:17.501595 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57622ea-72af-47f9-8218-9aee1791762c admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:17.513039 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57622ea-72af-47f9-8218-9aee1791762c admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:17.517921 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a59299-7d72-488e-9a6a-1885962efe1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:17.518563 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a59299-7d72-488e-9a6a-1885962efe1c admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:17.518874 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a59299-7d72-488e-9a6a-1885962efe1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:17.519151 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a59299-7d72-488e-9a6a-1885962efe1c admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:17.519859 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a59299-7d72-488e-9a6a-1885962efe1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:17.521515 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a59299-7d72-488e-9a6a-1885962efe1c admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:17.533161 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1118/2223] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:35:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:35:17.537313 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc9fb38-38cd-411d-913c-ab449a58b36e admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:17.547360 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc9fb38-38cd-411d-913c-ab449a58b36e admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:17.552083 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd60f807-86af-446c-b9d8-83006ac22dd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:17.552691 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd60f807-86af-446c-b9d8-83006ac22dd9 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:17.552974 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd60f807-86af-446c-b9d8-83006ac22dd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:17.553283 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd60f807-86af-446c-b9d8-83006ac22dd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:17.553969 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd60f807-86af-446c-b9d8-83006ac22dd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:17.556243 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd60f807-86af-446c-b9d8-83006ac22dd9 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:17.567002 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1106/2224] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:35:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 01:35:17.571901 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a59299-7d72-488e-9a6a-1885962efe1c admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:17.582903 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a59299-7d72-488e-9a6a-1885962efe1c admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:17.590714 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c632c2a-0c79-46e6-9a67-75fef8f0fe59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:17.591399 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c632c2a-0c79-46e6-9a67-75fef8f0fe59 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:17.591686 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c632c2a-0c79-46e6-9a67-75fef8f0fe59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:17.591964 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c632c2a-0c79-46e6-9a67-75fef8f0fe59 admin admin] PATH_INFO: `/v3/projects/8c46ce90b6074d188f6c9cf7b1596009/users/6ab71788e89f42c9bcccacbda29e8514/roles/61b3b93d37a047e4bb8369b46fe20db9` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:17.606927 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c632c2a-0c79-46e6-9a67-75fef8f0fe59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c46ce90b6074d188f6c9cf7b1596009, user_id=6ab71788e89f42c9bcccacbda29e8514, role_id=61b3b93d37a047e4bb8369b46fe20db9)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:17.609618 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c632c2a-0c79-46e6-9a67-75fef8f0fe59 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:17.624019 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1119/2225] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:35:17 2022] PUT /identity/v3/projects/8c46ce90b6074d188f6c9cf7b1596009/users/6ab71788e89f42c9bcccacbda29e8514/roles/61b3b93d37a047e4bb8369b46fe20db9 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:35:17.629752 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486e799b-be3d-45d0-bc5a-04c99195b59c None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:17.630041 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486e799b-be3d-45d0-bc5a-04c99195b59c None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:17.630483 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486e799b-be3d-45d0-bc5a-04c99195b59c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:17.650226 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-486e799b-be3d-45d0-bc5a-04c99195b59c None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:35:17.651450 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-486e799b-be3d-45d0-bc5a-04c99195b59c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:35:17.711705 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1107/2226] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:35:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:35:17.721841 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c632c2a-0c79-46e6-9a67-75fef8f0fe59 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:17.723623 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-2c632c2a-0c79-46e6-9a67-75fef8f0fe59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:35:17.729186 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:35:17.782828 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c632c2a-0c79-46e6-9a67-75fef8f0fe59 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:17.786668 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e867513-37c3-4bc4-b309-5bccea703d6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:17.787412 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e867513-37c3-4bc4-b309-5bccea703d6c service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:17.787727 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e867513-37c3-4bc4-b309-5bccea703d6c service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:17.788130 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e867513-37c3-4bc4-b309-5bccea703d6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:17.801291 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e867513-37c3-4bc4-b309-5bccea703d6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:17.802731 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e867513-37c3-4bc4-b309-5bccea703d6c service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:17.813107 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1120/2227] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:35:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:35:18.458922 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd9e586-8225-418a-9514-4dc6eb7ac89b None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:18.459776 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd9e586-8225-418a-9514-4dc6eb7ac89b None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:18.460132 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd9e586-8225-418a-9514-4dc6eb7ac89b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:18.492830 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-0fd9e586-8225-418a-9514-4dc6eb7ac89b None None] MFA Rules not processed for user `6ab71788e89f42c9bcccacbda29e8514`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:35:18.494867 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-0fd9e586-8225-418a-9514-4dc6eb7ac89b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:35:18.581322 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1108/2228] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:35:18 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:35:18.584930 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e867513-37c3-4bc4-b309-5bccea703d6c service neutron] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:18.599154 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e867513-37c3-4bc4-b309-5bccea703d6c service neutron] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:18.608578 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51740a9a-ba71-4695-83ab-4be363f6cb18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:18.609477 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51740a9a-ba71-4695-83ab-4be363f6cb18 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:18.610518 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51740a9a-ba71-4695-83ab-4be363f6cb18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:18.610802 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51740a9a-ba71-4695-83ab-4be363f6cb18 admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:18.611503 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51740a9a-ba71-4695-83ab-4be363f6cb18 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:18.615554 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51740a9a-ba71-4695-83ab-4be363f6cb18 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:18.647938 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1121/2229] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Aug 6 01:35:18 2022] POST /identity/v3/projects => generated 371 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:35:18.652178 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd9e586-8225-418a-9514-4dc6eb7ac89b None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:18.666565 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd9e586-8225-418a-9514-4dc6eb7ac89b None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:18.677382 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:35:18.728196 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e187915-a6b0-4c75-8ab1-6956bb8b74d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:18.728831 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e187915-a6b0-4c75-8ab1-6956bb8b74d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:18.729119 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e187915-a6b0-4c75-8ab1-6956bb8b74d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:18.729402 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e187915-a6b0-4c75-8ab1-6956bb8b74d5 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:18.730081 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e187915-a6b0-4c75-8ab1-6956bb8b74d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:18.732010 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e187915-a6b0-4c75-8ab1-6956bb8b74d5 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:18.765454 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1109/2230] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:35:18 2022] POST /identity/v3/users => generated 292 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:35:18.769236 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51740a9a-ba71-4695-83ab-4be363f6cb18 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:18.781479 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51740a9a-ba71-4695-83ab-4be363f6cb18 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:18.786473 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9ec8b9-88b3-4b22-9142-0834581957bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:18.787097 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9ec8b9-88b3-4b22-9142-0834581957bb admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:18.787404 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9ec8b9-88b3-4b22-9142-0834581957bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:18.787681 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9ec8b9-88b3-4b22-9142-0834581957bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:18.788382 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9ec8b9-88b3-4b22-9142-0834581957bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:18.790571 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9ec8b9-88b3-4b22-9142-0834581957bb admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:18.802462 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1122/2231] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:35:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:35:18.807442 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e187915-a6b0-4c75-8ab1-6956bb8b74d5 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:18.818013 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e187915-a6b0-4c75-8ab1-6956bb8b74d5 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:18.823185 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf5e858-aca0-4118-b091-0339428ccb8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:18.823926 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf5e858-aca0-4118-b091-0339428ccb8d admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:18.824275 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf5e858-aca0-4118-b091-0339428ccb8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:18.824642 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf5e858-aca0-4118-b091-0339428ccb8d admin admin] PATH_INFO: `/v3/projects/a12c45b3799844929c283646e5aaf539/users/1dbafeae63a54d82a1b03d2e5f6806c6/roles/4559b9a0837249f895a13c70096f8d38` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:18.841444 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf5e858-aca0-4118-b091-0339428ccb8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a12c45b3799844929c283646e5aaf539, user_id=1dbafeae63a54d82a1b03d2e5f6806c6, role_id=4559b9a0837249f895a13c70096f8d38)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:18.844471 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf5e858-aca0-4118-b091-0339428ccb8d admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:18.863117 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1110/2232] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:35:18 2022] PUT /identity/v3/projects/a12c45b3799844929c283646e5aaf539/users/1dbafeae63a54d82a1b03d2e5f6806c6/roles/4559b9a0837249f895a13c70096f8d38 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:35:18.868091 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9ec8b9-88b3-4b22-9142-0834581957bb admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:18.879316 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9ec8b9-88b3-4b22-9142-0834581957bb admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:18.950253 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc28d03a-1d88-41c8-8d41-4f5ee7c01f5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:18.950891 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc28d03a-1d88-41c8-8d41-4f5ee7c01f5b admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:18.951175 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc28d03a-1d88-41c8-8d41-4f5ee7c01f5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:18.951457 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc28d03a-1d88-41c8-8d41-4f5ee7c01f5b admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:18.952170 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc28d03a-1d88-41c8-8d41-4f5ee7c01f5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:18.957090 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc28d03a-1d88-41c8-8d41-4f5ee7c01f5b admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:18.968500 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1123/2233] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:35:18 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:35:18.969756 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf5e858-aca0-4118-b091-0339428ccb8d admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:18.981198 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf5e858-aca0-4118-b091-0339428ccb8d admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:18.986041 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db66421c-f1f8-4445-b82c-02949d9647d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:18.986714 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db66421c-f1f8-4445-b82c-02949d9647d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:18.986994 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db66421c-f1f8-4445-b82c-02949d9647d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:18.987271 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db66421c-f1f8-4445-b82c-02949d9647d6 admin admin] PATH_INFO: `/v3/projects/a12c45b3799844929c283646e5aaf539/users/1dbafeae63a54d82a1b03d2e5f6806c6/roles/4559b9a0837249f895a13c70096f8d38` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:18.989265 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db66421c-f1f8-4445-b82c-02949d9647d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a12c45b3799844929c283646e5aaf539, user_id=1dbafeae63a54d82a1b03d2e5f6806c6, role_id=4559b9a0837249f895a13c70096f8d38)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:18.992993 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db66421c-f1f8-4445-b82c-02949d9647d6 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:19.004558 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1111/2234] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:35:18 2022] PUT /identity/v3/projects/a12c45b3799844929c283646e5aaf539/users/1dbafeae63a54d82a1b03d2e5f6806c6/roles/4559b9a0837249f895a13c70096f8d38 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:35:20.000004 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b39882-dd81-43a4-a3c7-fea1d9a7d2cf None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:20.000549 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b39882-dd81-43a4-a3c7-fea1d9a7d2cf None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:20.000549 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b39882-dd81-43a4-a3c7-fea1d9a7d2cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:20.031532 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-a6b39882-dd81-43a4-a3c7-fea1d9a7d2cf None None] MFA Rules not processed for user `1dbafeae63a54d82a1b03d2e5f6806c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:35:20.032756 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-a6b39882-dd81-43a4-a3c7-fea1d9a7d2cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:35:20.082574 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a8bffa-a345-4969-9b47-e4ad4a29c983 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:20.083036 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a8bffa-a345-4969-9b47-e4ad4a29c983 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:20.083877 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a8bffa-a345-4969-9b47-e4ad4a29c983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:20.104950 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-78a8bffa-a345-4969-9b47-e4ad4a29c983 None None] MFA Rules not processed for user `37c704ce557c4f309cf00ef32646235f`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:35:20.106560 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-78a8bffa-a345-4969-9b47-e4ad4a29c983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:35:20.113333 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:35:20.126495 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1124/2235] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:35:19 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:35:20.140569 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b39882-dd81-43a4-a3c7-fea1d9a7d2cf None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:20.142944 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-a6b39882-dd81-43a4-a3c7-fea1d9a7d2cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:35:20.173920 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1112/2236] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:35:20 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:35:20.212827 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b39882-dd81-43a4-a3c7-fea1d9a7d2cf None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:20.216431 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48b213b-1fb2-407b-8ad1-4a980e69206b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1be1e6173e374a60b9e1fbf51a1ab571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:20.217069 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48b213b-1fb2-407b-8ad1-4a980e69206b service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:20.217373 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48b213b-1fb2-407b-8ad1-4a980e69206b service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:20.217654 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48b213b-1fb2-407b-8ad1-4a980e69206b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:20.284497 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48b213b-1fb2-407b-8ad1-4a980e69206b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:20.286000 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48b213b-1fb2-407b-8ad1-4a980e69206b service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:20.296256 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1125/2237] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:35:20 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 01:35:29.481941 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8342060a-b46f-4e74-9f3e-54fcbc599de1 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:29.482468 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8342060a-b46f-4e74-9f3e-54fcbc599de1 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:29.482614 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8342060a-b46f-4e74-9f3e-54fcbc599de1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:29.503200 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-8342060a-b46f-4e74-9f3e-54fcbc599de1 None None] MFA Rules not processed for user `8e86f38590824894a3b960e6774ff8a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:35:29.504558 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-8342060a-b46f-4e74-9f3e-54fcbc599de1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:35:29.564131 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1113/2238] 10.222.0.26 () {58 vars in 1009 bytes} [Sat Aug 6 01:35:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:35:29.606885 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48b213b-1fb2-407b-8ad1-4a980e69206b service cinder] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:29.608435 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-e48b213b-1fb2-407b-8ad1-4a980e69206b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:35:29.684769 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48b213b-1fb2-407b-8ad1-4a980e69206b service cinder] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:29.690148 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fd9a35-f315-4ec7-bf61-af14a6adc0ba service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b23f4478f49471fba6f43e6e4be99da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:29.690776 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fd9a35-f315-4ec7-bf61-af14a6adc0ba service swift] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:29.691069 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fd9a35-f315-4ec7-bf61-af14a6adc0ba service swift] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:29.691345 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fd9a35-f315-4ec7-bf61-af14a6adc0ba service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:29.700691 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fd9a35-f315-4ec7-bf61-af14a6adc0ba service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:29.702321 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fd9a35-f315-4ec7-bf61-af14a6adc0ba service swift] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:29.712069 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1126/2239] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Aug 6 01:35:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 06 01:35:30.154462 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8342060a-b46f-4e74-9f3e-54fcbc599de1 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:30.165634 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8342060a-b46f-4e74-9f3e-54fcbc599de1 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:30.169803 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7e4ebe-29fe-49f7-b0fa-4e31d42d17bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1be1e6173e374a60b9e1fbf51a1ab571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:30.170774 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7e4ebe-29fe-49f7-b0fa-4e31d42d17bb service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:30.171282 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7e4ebe-29fe-49f7-b0fa-4e31d42d17bb service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:30.171746 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7e4ebe-29fe-49f7-b0fa-4e31d42d17bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:30.191248 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7e4ebe-29fe-49f7-b0fa-4e31d42d17bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:30.202631 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7e4ebe-29fe-49f7-b0fa-4e31d42d17bb service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:30.224009 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1114/2240] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:35:30 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:35:39.707637 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=83 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:35:40.215473 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=94,keystone:INSERT=5 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:35:50.395130 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf50da7-d98c-46d5-9e9c-1019cd297a99 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:50.395698 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf50da7-d98c-46d5-9e9c-1019cd297a99 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:50.395698 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf50da7-d98c-46d5-9e9c-1019cd297a99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:50.415309 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-bdf50da7-d98c-46d5-9e9c-1019cd297a99 None None] MFA Rules not processed for user `8e86f38590824894a3b960e6774ff8a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:35:50.416545 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-bdf50da7-d98c-46d5-9e9c-1019cd297a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:35:50.423697 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1127/2241] 10.222.0.26 () {58 vars in 967 bytes} [Sat Aug 6 01:35:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:35:50.436984 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7e4ebe-29fe-49f7-b0fa-4e31d42d17bb service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:50.448419 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7e4ebe-29fe-49f7-b0fa-4e31d42d17bb service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:50.453785 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9942f5-60dd-4109-8646-fea3151c3aaa service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b23f4478f49471fba6f43e6e4be99da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:50.454407 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9942f5-60dd-4109-8646-fea3151c3aaa service swift] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:50.454691 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9942f5-60dd-4109-8646-fea3151c3aaa service swift] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:50.454972 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9942f5-60dd-4109-8646-fea3151c3aaa service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:50.463759 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9942f5-60dd-4109-8646-fea3151c3aaa service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:50.465405 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9942f5-60dd-4109-8646-fea3151c3aaa service swift] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:50.473506 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1115/2242] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Aug 6 01:35:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 06 01:35:53.879509 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e0d0d6-836e-48f2-9097-7b44a294d77f None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:53.880045 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e0d0d6-836e-48f2-9097-7b44a294d77f None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:53.880045 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e0d0d6-836e-48f2-9097-7b44a294d77f None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:53.881458 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1128/2243] 10.222.0.26 () {54 vars in 921 bytes} [Sat Aug 6 01:35:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:35:53.917233 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b15d3ab-9edf-4d55-9d89-692b43a05cef None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:53.917732 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b15d3ab-9edf-4d55-9d89-692b43a05cef None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:53.917823 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b15d3ab-9edf-4d55-9d89-692b43a05cef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:53.937911 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-0b15d3ab-9edf-4d55-9d89-692b43a05cef None None] MFA Rules not processed for user `fa3b93ddd9464a2cba724d67b56b477a`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:35:53.938813 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-0b15d3ab-9edf-4d55-9d89-692b43a05cef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:35:53.989173 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e0d0d6-836e-48f2-9097-7b44a294d77f None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:53.991709 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-43e0d0d6-836e-48f2-9097-7b44a294d77f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:35:54.023236 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1116/2244] 10.222.0.26 () {58 vars in 1018 bytes} [Sat Aug 6 01:35:53 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:35:54.042830 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b15d3ab-9edf-4d55-9d89-692b43a05cef None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:54.045381 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-0b15d3ab-9edf-4d55-9d89-692b43a05cef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:35:54.059654 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b15d3ab-9edf-4d55-9d89-692b43a05cef None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:54.066213 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e2b3ef-df0b-42f7-b8fe-00a37bb3008e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa3b93ddd9464a2cba724d67b56b477a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:54.066833 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e2b3ef-df0b-42f7-b8fe-00a37bb3008e service placement] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:54.067117 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e2b3ef-df0b-42f7-b8fe-00a37bb3008e service placement] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:54.067393 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e2b3ef-df0b-42f7-b8fe-00a37bb3008e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:54.068420 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e0d0d6-836e-48f2-9097-7b44a294d77f None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:54.074660 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccfb92a-ab3d-4e59-9c90-ea16c1df57fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:54.075697 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccfb92a-ab3d-4e59-9c90-ea16c1df57fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:54.075697 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccfb92a-ab3d-4e59-9c90-ea16c1df57fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:54.075880 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccfb92a-ab3d-4e59-9c90-ea16c1df57fc admin admin] PATH_INFO: `/v3/users/3f081e5336f74e21903f11df0b9d95b1` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:54.077065 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccfb92a-ab3d-4e59-9c90-ea16c1df57fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f081e5336f74e21903f11df0b9d95b1)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:54.078356 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e2b3ef-df0b-42f7-b8fe-00a37bb3008e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:54.079106 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccfb92a-ab3d-4e59-9c90-ea16c1df57fc admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:54.079953 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e2b3ef-df0b-42f7-b8fe-00a37bb3008e service placement] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:54.091341 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1117/2245] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:35:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 01:35:54.136214 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-1ccfb92a-ab3d-4e59-9c90-ea16c1df57fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f081e5336f74e21903f11df0b9d95b1'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:54.144217 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-1ccfb92a-ab3d-4e59-9c90-ea16c1df57fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f081e5336f74e21903f11df0b9d95b1'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:54.144987 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-1ccfb92a-ab3d-4e59-9c90-ea16c1df57fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f081e5336f74e21903f11df0b9d95b1'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:54.145665 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-1ccfb92a-ab3d-4e59-9c90-ea16c1df57fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f081e5336f74e21903f11df0b9d95b1'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:54.159328 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-1ccfb92a-ab3d-4e59-9c90-ea16c1df57fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f081e5336f74e21903f11df0b9d95b1'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:54.173712 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1129/2246] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Aug 6 01:35:53 2022] DELETE /identity/v3/users/3f081e5336f74e21903f11df0b9d95b1 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:35:54.426162 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e2b3ef-df0b-42f7-b8fe-00a37bb3008e service placement] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:54.441073 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e2b3ef-df0b-42f7-b8fe-00a37bb3008e service placement] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:54.530495 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b740fc2-19dc-4dd8-b419-afc2d2c3c7a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:54.531380 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b740fc2-19dc-4dd8-b419-afc2d2c3c7a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:54.531857 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b740fc2-19dc-4dd8-b419-afc2d2c3c7a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:54.532968 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b740fc2-19dc-4dd8-b419-afc2d2c3c7a0 admin admin] PATH_INFO: `/v3/projects/d37acbae2bf64dac8eec9721189268d0` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:54.534337 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b740fc2-19dc-4dd8-b419-afc2d2c3c7a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d37acbae2bf64dac8eec9721189268d0)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:54.538570 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b740fc2-19dc-4dd8-b419-afc2d2c3c7a0 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:54.622280 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-6b740fc2-19dc-4dd8-b419-afc2d2c3c7a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd37acbae2bf64dac8eec9721189268d0', 'request_id': 'req-6b740fc2-19dc-4dd8-b419-afc2d2c3c7a0'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:54.637515 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-6b740fc2-19dc-4dd8-b419-afc2d2c3c7a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd37acbae2bf64dac8eec9721189268d0', 'request_id': 'req-6b740fc2-19dc-4dd8-b419-afc2d2c3c7a0'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:54.644178 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-6b740fc2-19dc-4dd8-b419-afc2d2c3c7a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd37acbae2bf64dac8eec9721189268d0', 'request_id': 'req-6b740fc2-19dc-4dd8-b419-afc2d2c3c7a0'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:54.660328 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:35:54.670907 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-6b740fc2-19dc-4dd8-b419-afc2d2c3c7a0 admin admin] The token cache is being invalidate because project d37acbae2bf64dac8eec9721189268d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:35:54.671358 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-6b740fc2-19dc-4dd8-b419-afc2d2c3c7a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:54.672393 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-6b740fc2-19dc-4dd8-b419-afc2d2c3c7a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:54.681127 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1118/2247] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Aug 6 01:35:54 2022] DELETE /identity/v3/projects/d37acbae2bf64dac8eec9721189268d0 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:35:54.684152 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c913551-b8a5-4df8-abc7-2f74200191a7 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:54.684445 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c913551-b8a5-4df8-abc7-2f74200191a7 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:54.684721 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c913551-b8a5-4df8-abc7-2f74200191a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:54.737283 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-2c913551-b8a5-4df8-abc7-2f74200191a7 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:35:54.738389 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-2c913551-b8a5-4df8-abc7-2f74200191a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:35:54.862656 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1130/2248] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:35:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:35:54.873952 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f26524-8c51-4b6f-851c-41a4f5d385e3 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:54.874165 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f26524-8c51-4b6f-851c-41a4f5d385e3 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:54.874449 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f26524-8c51-4b6f-851c-41a4f5d385e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:54.899391 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-29f26524-8c51-4b6f-851c-41a4f5d385e3 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:35:54.901154 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-29f26524-8c51-4b6f-851c-41a4f5d385e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:35:54.910523 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1119/2249] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:35:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:35:54.918026 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c913551-b8a5-4df8-abc7-2f74200191a7 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:54.928608 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c913551-b8a5-4df8-abc7-2f74200191a7 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:54.933839 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123b0859-acf3-450c-b619-02fe3ac736a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:54.934453 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123b0859-acf3-450c-b619-02fe3ac736a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:54.934915 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123b0859-acf3-450c-b619-02fe3ac736a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:54.935188 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123b0859-acf3-450c-b619-02fe3ac736a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:54.935890 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123b0859-acf3-450c-b619-02fe3ac736a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:35:54.936198 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123b0859-acf3-450c-b619-02fe3ac736a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:54.937858 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123b0859-acf3-450c-b619-02fe3ac736a4 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:54.946695 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:35:54.955553 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1131/2250] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Aug 6 01:35:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:35:54.956564 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f26524-8c51-4b6f-851c-41a4f5d385e3 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:54.966576 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f26524-8c51-4b6f-851c-41a4f5d385e3 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:54.971361 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a753cd-9132-4214-a0c5-a9e3c570ecbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:54.971992 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a753cd-9132-4214-a0c5-a9e3c570ecbb admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:54.972433 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a753cd-9132-4214-a0c5-a9e3c570ecbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:54.972544 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a753cd-9132-4214-a0c5-a9e3c570ecbb admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:54.974105 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a753cd-9132-4214-a0c5-a9e3c570ecbb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:54.975471 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a753cd-9132-4214-a0c5-a9e3c570ecbb admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:55.007873 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1120/2251] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Aug 6 01:35:54 2022] POST /identity/v3/projects => generated 361 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:35:55.013159 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123b0859-acf3-450c-b619-02fe3ac736a4 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:55.023547 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123b0859-acf3-450c-b619-02fe3ac736a4 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:55.088052 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef0a5d4-a516-46e5-b8ab-2eed13895f07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:55.088627 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef0a5d4-a516-46e5-b8ab-2eed13895f07 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:55.088929 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef0a5d4-a516-46e5-b8ab-2eed13895f07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:55.089257 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef0a5d4-a516-46e5-b8ab-2eed13895f07 admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:55.089904 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef0a5d4-a516-46e5-b8ab-2eed13895f07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:55.091871 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef0a5d4-a516-46e5-b8ab-2eed13895f07 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:55.118043 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1132/2252] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:35:55 2022] POST /identity/v3/users => generated 281 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:35:55.122775 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a753cd-9132-4214-a0c5-a9e3c570ecbb admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:55.138183 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a753cd-9132-4214-a0c5-a9e3c570ecbb admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:55.143344 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb99ba0-fe42-470d-9b39-4004bea16d37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:55.144104 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb99ba0-fe42-470d-9b39-4004bea16d37 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:55.144593 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb99ba0-fe42-470d-9b39-4004bea16d37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:55.144718 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb99ba0-fe42-470d-9b39-4004bea16d37 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:55.145466 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb99ba0-fe42-470d-9b39-4004bea16d37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:55.147292 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb99ba0-fe42-470d-9b39-4004bea16d37 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:55.160271 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1121/2253] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:35:55 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:35:55.165929 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef0a5d4-a516-46e5-b8ab-2eed13895f07 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:55.177277 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef0a5d4-a516-46e5-b8ab-2eed13895f07 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:55.182350 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518a8bbc-d8e0-4d33-bcfa-a5f0afa13614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:55.182965 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518a8bbc-d8e0-4d33-bcfa-a5f0afa13614 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:55.183245 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518a8bbc-d8e0-4d33-bcfa-a5f0afa13614 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:55.183516 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518a8bbc-d8e0-4d33-bcfa-a5f0afa13614 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:55.184212 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518a8bbc-d8e0-4d33-bcfa-a5f0afa13614 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:55.185847 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518a8bbc-d8e0-4d33-bcfa-a5f0afa13614 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:55.200195 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1133/2254] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:35:55 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:35:55.201629 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb99ba0-fe42-470d-9b39-4004bea16d37 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:55.212433 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb99ba0-fe42-470d-9b39-4004bea16d37 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:55.217879 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c915d70-c766-4260-a114-73904f8a416b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:55.219234 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c915d70-c766-4260-a114-73904f8a416b admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:55.219527 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c915d70-c766-4260-a114-73904f8a416b admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:55.219805 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c915d70-c766-4260-a114-73904f8a416b admin admin] PATH_INFO: `/v3/projects/86651f48c1f441f1b0e115e57b840a28/users/7b41c8b1389445e59b0d5f388873ec7f/roles/61b3b93d37a047e4bb8369b46fe20db9` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:55.237210 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c915d70-c766-4260-a114-73904f8a416b admin admin] RBAC: Authorizing `identity:create_grant(project_id=86651f48c1f441f1b0e115e57b840a28, user_id=7b41c8b1389445e59b0d5f388873ec7f, role_id=61b3b93d37a047e4bb8369b46fe20db9)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:55.240169 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c915d70-c766-4260-a114-73904f8a416b admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:55.255476 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1122/2255] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:35:55 2022] PUT /identity/v3/projects/86651f48c1f441f1b0e115e57b840a28/users/7b41c8b1389445e59b0d5f388873ec7f/roles/61b3b93d37a047e4bb8369b46fe20db9 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:35:55.266779 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f1d4cb-b303-4928-8ea9-07147562f46c None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:55.267329 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f1d4cb-b303-4928-8ea9-07147562f46c None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:55.267675 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f1d4cb-b303-4928-8ea9-07147562f46c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:55.298875 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-57f1d4cb-b303-4928-8ea9-07147562f46c None None] MFA Rules not processed for user `7b41c8b1389445e59b0d5f388873ec7f`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:35:55.300090 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-57f1d4cb-b303-4928-8ea9-07147562f46c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:35:55.371958 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1134/2256] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:35:55 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:35:55.388130 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c915d70-c766-4260-a114-73904f8a416b admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:55.390094 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-3c915d70-c766-4260-a114-73904f8a416b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:35:55.477969 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c915d70-c766-4260-a114-73904f8a416b admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:55.481747 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbb825d-b48c-4953-9598-4ee045ad68b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1be1e6173e374a60b9e1fbf51a1ab571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:55.482604 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbb825d-b48c-4953-9598-4ee045ad68b2 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:55.482895 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbb825d-b48c-4953-9598-4ee045ad68b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:55.483184 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbb825d-b48c-4953-9598-4ee045ad68b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:55.492170 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbb825d-b48c-4953-9598-4ee045ad68b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:55.493721 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbb825d-b48c-4953-9598-4ee045ad68b2 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:55.502559 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1123/2257] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:35:55 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:36:05.212913 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f1d4cb-b303-4928-8ea9-07147562f46c None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:36:05.214406 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-57f1d4cb-b303-4928-8ea9-07147562f46c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:36:05.281068 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f1d4cb-b303-4928-8ea9-07147562f46c None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:36:05.285573 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13187d07-a275-44b1-a989-4aa8d6f0a9e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:36:05.286206 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13187d07-a275-44b1-a989-4aa8d6f0a9e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:36:05.286489 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13187d07-a275-44b1-a989-4aa8d6f0a9e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:36:05.286765 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13187d07-a275-44b1-a989-4aa8d6f0a9e6 admin admin] PATH_INFO: `/v3/users/6ab71788e89f42c9bcccacbda29e8514` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:36:05.287919 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13187d07-a275-44b1-a989-4aa8d6f0a9e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ab71788e89f42c9bcccacbda29e8514)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:36:05.289864 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13187d07-a275-44b1-a989-4aa8d6f0a9e6 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:36:05.300494 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:36:05.335401 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-13187d07-a275-44b1-a989-4aa8d6f0a9e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ab71788e89f42c9bcccacbda29e8514'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:36:05.342731 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-13187d07-a275-44b1-a989-4aa8d6f0a9e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ab71788e89f42c9bcccacbda29e8514'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:36:05.343391 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-13187d07-a275-44b1-a989-4aa8d6f0a9e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ab71788e89f42c9bcccacbda29e8514'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:36:05.343999 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-13187d07-a275-44b1-a989-4aa8d6f0a9e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ab71788e89f42c9bcccacbda29e8514'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:36:05.355849 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-13187d07-a275-44b1-a989-4aa8d6f0a9e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ab71788e89f42c9bcccacbda29e8514'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:36:05.368499 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1135/2258] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Aug 6 01:36:05 2022] DELETE /identity/v3/users/6ab71788e89f42c9bcccacbda29e8514 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:36:05.498552 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=53,keystone:INSERT=2 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:36:05.540312 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbb825d-b48c-4953-9598-4ee045ad68b2 service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:36:05.552141 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbb825d-b48c-4953-9598-4ee045ad68b2 service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:36:05.618739 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7921e6-eb65-467c-8bb3-24a27e8305f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:36:05.619305 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7921e6-eb65-467c-8bb3-24a27e8305f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:36:05.619585 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7921e6-eb65-467c-8bb3-24a27e8305f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:36:05.619860 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7921e6-eb65-467c-8bb3-24a27e8305f2 admin admin] PATH_INFO: `/v3/projects/8c46ce90b6074d188f6c9cf7b1596009` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:36:05.621067 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7921e6-eb65-467c-8bb3-24a27e8305f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c46ce90b6074d188f6c9cf7b1596009)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:36:05.623608 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7921e6-eb65-467c-8bb3-24a27e8305f2 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:36:05.675562 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-2f7921e6-eb65-467c-8bb3-24a27e8305f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c46ce90b6074d188f6c9cf7b1596009', 'request_id': 'req-2f7921e6-eb65-467c-8bb3-24a27e8305f2'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:36:05.687859 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-2f7921e6-eb65-467c-8bb3-24a27e8305f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c46ce90b6074d188f6c9cf7b1596009', 'request_id': 'req-2f7921e6-eb65-467c-8bb3-24a27e8305f2'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:36:05.693693 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-2f7921e6-eb65-467c-8bb3-24a27e8305f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c46ce90b6074d188f6c9cf7b1596009', 'request_id': 'req-2f7921e6-eb65-467c-8bb3-24a27e8305f2'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:36:05.708003 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-2f7921e6-eb65-467c-8bb3-24a27e8305f2 admin admin] The token cache is being invalidate because project 8c46ce90b6074d188f6c9cf7b1596009 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:36:05.708386 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-2f7921e6-eb65-467c-8bb3-24a27e8305f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:36:05.709162 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-2f7921e6-eb65-467c-8bb3-24a27e8305f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:36:05.710773 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1124/2259] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Aug 6 01:36:05 2022] DELETE /identity/v3/projects/8c46ce90b6074d188f6c9cf7b1596009 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:36:05.714969 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13187d07-a275-44b1-a989-4aa8d6f0a9e6 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:36:05.716532 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-13187d07-a275-44b1-a989-4aa8d6f0a9e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:36:05.729793 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13187d07-a275-44b1-a989-4aa8d6f0a9e6 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:36:05.786248 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2f6417-9411-47c0-9aba-499642b3b2df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:36:05.786879 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2f6417-9411-47c0-9aba-499642b3b2df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:36:05.787160 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2f6417-9411-47c0-9aba-499642b3b2df admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:36:05.787435 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2f6417-9411-47c0-9aba-499642b3b2df admin admin] PATH_INFO: `/v3/users/1dbafeae63a54d82a1b03d2e5f6806c6` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:36:05.788586 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2f6417-9411-47c0-9aba-499642b3b2df admin admin] RBAC: Authorizing `identity:delete_user(user_id=1dbafeae63a54d82a1b03d2e5f6806c6)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:36:05.790520 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2f6417-9411-47c0-9aba-499642b3b2df admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:36:05.837813 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-2a2f6417-9411-47c0-9aba-499642b3b2df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1dbafeae63a54d82a1b03d2e5f6806c6'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:36:05.844641 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-2a2f6417-9411-47c0-9aba-499642b3b2df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1dbafeae63a54d82a1b03d2e5f6806c6'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:36:05.845331 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-2a2f6417-9411-47c0-9aba-499642b3b2df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1dbafeae63a54d82a1b03d2e5f6806c6'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:36:05.845949 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-2a2f6417-9411-47c0-9aba-499642b3b2df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1dbafeae63a54d82a1b03d2e5f6806c6'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:36:05.858343 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-2a2f6417-9411-47c0-9aba-499642b3b2df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1dbafeae63a54d82a1b03d2e5f6806c6'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:36:05.870513 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1136/2260] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Aug 6 01:36:05 2022] DELETE /identity/v3/users/1dbafeae63a54d82a1b03d2e5f6806c6 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:36:06.050244 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7921e6-eb65-467c-8bb3-24a27e8305f2 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:36:06.051897 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-2f7921e6-eb65-467c-8bb3-24a27e8305f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:36:06.066490 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7921e6-eb65-467c-8bb3-24a27e8305f2 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:36:06.124458 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e473d5ba-5eee-483c-a20e-14f715d0b767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:36:06.125025 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e473d5ba-5eee-483c-a20e-14f715d0b767 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:36:06.125408 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e473d5ba-5eee-483c-a20e-14f715d0b767 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:36:06.125688 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e473d5ba-5eee-483c-a20e-14f715d0b767 admin admin] PATH_INFO: `/v3/projects/a12c45b3799844929c283646e5aaf539` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:36:06.126849 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e473d5ba-5eee-483c-a20e-14f715d0b767 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a12c45b3799844929c283646e5aaf539)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:36:06.128754 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e473d5ba-5eee-483c-a20e-14f715d0b767 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:36:06.164477 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:36:06.178608 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-e473d5ba-5eee-483c-a20e-14f715d0b767 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a12c45b3799844929c283646e5aaf539', 'request_id': 'req-e473d5ba-5eee-483c-a20e-14f715d0b767'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:36:06.184792 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-e473d5ba-5eee-483c-a20e-14f715d0b767 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a12c45b3799844929c283646e5aaf539', 'request_id': 'req-e473d5ba-5eee-483c-a20e-14f715d0b767'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:36:06.190266 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-e473d5ba-5eee-483c-a20e-14f715d0b767 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a12c45b3799844929c283646e5aaf539', 'request_id': 'req-e473d5ba-5eee-483c-a20e-14f715d0b767'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:36:06.202432 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-e473d5ba-5eee-483c-a20e-14f715d0b767 admin admin] The token cache is being invalidate because project a12c45b3799844929c283646e5aaf539 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:36:06.202724 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-e473d5ba-5eee-483c-a20e-14f715d0b767 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:36:06.203383 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-e473d5ba-5eee-483c-a20e-14f715d0b767 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:36:06.204915 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1125/2261] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Aug 6 01:36:06 2022] DELETE /identity/v3/projects/a12c45b3799844929c283646e5aaf539 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:36:06.209965 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14538125-9f0f-4b35-b1cb-0116b962d8d3 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:36:06.210358 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14538125-9f0f-4b35-b1cb-0116b962d8d3 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:36:06.210649 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14538125-9f0f-4b35-b1cb-0116b962d8d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:36:06.225423 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:DELETE=20,keystone:SELECT=78,keystone:INSERT=2 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:36:06.233182 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-14538125-9f0f-4b35-b1cb-0116b962d8d3 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:36:06.234908 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-14538125-9f0f-4b35-b1cb-0116b962d8d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:36:06.294819 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1137/2262] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:36:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:36:06.302594 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eedc18f-f6ba-4ecd-b95d-061305acc8fc None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:36:06.302888 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eedc18f-f6ba-4ecd-b95d-061305acc8fc None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:36:06.303191 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eedc18f-f6ba-4ecd-b95d-061305acc8fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:36:06.323031 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-8eedc18f-f6ba-4ecd-b95d-061305acc8fc None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:36:06.324243 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-8eedc18f-f6ba-4ecd-b95d-061305acc8fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:36:06.332990 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1126/2263] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:36:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:36:06.337316 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14538125-9f0f-4b35-b1cb-0116b962d8d3 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:36:06.347184 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14538125-9f0f-4b35-b1cb-0116b962d8d3 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:36:06.351711 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b25caef-87de-41d1-8e1f-3e89a5eac425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:36:06.352308 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b25caef-87de-41d1-8e1f-3e89a5eac425 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:36:06.352584 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b25caef-87de-41d1-8e1f-3e89a5eac425 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:36:06.352857 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b25caef-87de-41d1-8e1f-3e89a5eac425 admin admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:36:06.353583 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b25caef-87de-41d1-8e1f-3e89a5eac425 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:36:06.353887 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b25caef-87de-41d1-8e1f-3e89a5eac425 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:36:06.355457 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b25caef-87de-41d1-8e1f-3e89a5eac425 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:36:06.367166 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1138/2264] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Aug 6 01:36:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:36:06.372138 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eedc18f-f6ba-4ecd-b95d-061305acc8fc None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:36:06.381564 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eedc18f-f6ba-4ecd-b95d-061305acc8fc None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:36:06.386190 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2446ed08-013b-4ad3-9f48-7c8a6a62386b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:36:06.387395 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2446ed08-013b-4ad3-9f48-7c8a6a62386b admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:36:06.387698 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2446ed08-013b-4ad3-9f48-7c8a6a62386b admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:36:06.387980 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2446ed08-013b-4ad3-9f48-7c8a6a62386b admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:36:06.388646 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2446ed08-013b-4ad3-9f48-7c8a6a62386b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:36:06.390659 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2446ed08-013b-4ad3-9f48-7c8a6a62386b admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:36:06.418427 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1127/2265] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Aug 6 01:36:06 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:36:06.422514 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b25caef-87de-41d1-8e1f-3e89a5eac425 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:36:06.432465 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b25caef-87de-41d1-8e1f-3e89a5eac425 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:36:06.487885 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04210eb-f006-4f30-b5d0-a99a106810ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:36:06.488494 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04210eb-f006-4f30-b5d0-a99a106810ab admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:36:06.488778 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04210eb-f006-4f30-b5d0-a99a106810ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:36:06.489076 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04210eb-f006-4f30-b5d0-a99a106810ab admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:36:06.489750 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04210eb-f006-4f30-b5d0-a99a106810ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:36:06.491936 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04210eb-f006-4f30-b5d0-a99a106810ab admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:36:06.513538 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1139/2266] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:36:06 2022] POST /identity/v3/users => generated 293 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:36:06.517435 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2446ed08-013b-4ad3-9f48-7c8a6a62386b admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:36:06.527514 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2446ed08-013b-4ad3-9f48-7c8a6a62386b admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:36:06.532588 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5e1bfd-6de7-4216-9568-707d9e2c6843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:36:06.533227 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5e1bfd-6de7-4216-9568-707d9e2c6843 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:36:06.533509 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5e1bfd-6de7-4216-9568-707d9e2c6843 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:36:06.533782 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5e1bfd-6de7-4216-9568-707d9e2c6843 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:36:06.534467 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5e1bfd-6de7-4216-9568-707d9e2c6843 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:36:06.536048 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5e1bfd-6de7-4216-9568-707d9e2c6843 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:36:06.546704 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1128/2267] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:36:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:36:06.550581 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04210eb-f006-4f30-b5d0-a99a106810ab admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:36:06.560938 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04210eb-f006-4f30-b5d0-a99a106810ab admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:36:06.565604 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caf0873-efa3-4dfb-a678-e9292878c835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:36:06.566205 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caf0873-efa3-4dfb-a678-e9292878c835 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:36:06.566492 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caf0873-efa3-4dfb-a678-e9292878c835 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:36:06.566786 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caf0873-efa3-4dfb-a678-e9292878c835 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:36:06.567477 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caf0873-efa3-4dfb-a678-e9292878c835 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:36:06.569087 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caf0873-efa3-4dfb-a678-e9292878c835 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:36:06.581343 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1140/2268] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:36:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 01:36:06.585284 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5e1bfd-6de7-4216-9568-707d9e2c6843 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:36:06.595767 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5e1bfd-6de7-4216-9568-707d9e2c6843 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:36:06.600460 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff05174-592f-4ab3-8ac5-3759dcb3dcb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:36:06.601143 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff05174-592f-4ab3-8ac5-3759dcb3dcb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:36:06.601426 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff05174-592f-4ab3-8ac5-3759dcb3dcb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:36:06.601703 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff05174-592f-4ab3-8ac5-3759dcb3dcb6 admin admin] PATH_INFO: `/v3/projects/e38b52a908c9435e843fde297df84b8b/users/1db83b7bf26f40079752392f6259caf8/roles/61b3b93d37a047e4bb8369b46fe20db9` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:36:06.617384 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff05174-592f-4ab3-8ac5-3759dcb3dcb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e38b52a908c9435e843fde297df84b8b, user_id=1db83b7bf26f40079752392f6259caf8, role_id=61b3b93d37a047e4bb8369b46fe20db9)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:36:06.620592 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff05174-592f-4ab3-8ac5-3759dcb3dcb6 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:36:06.634335 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1129/2269] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:36:06 2022] PUT /identity/v3/projects/e38b52a908c9435e843fde297df84b8b/users/1db83b7bf26f40079752392f6259caf8/roles/61b3b93d37a047e4bb8369b46fe20db9 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:36:06.646003 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7521e5b3-1e9b-48d2-b425-d9fc4603fc4b None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:36:06.646297 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7521e5b3-1e9b-48d2-b425-d9fc4603fc4b None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:36:06.646602 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7521e5b3-1e9b-48d2-b425-d9fc4603fc4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:36:06.674873 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-7521e5b3-1e9b-48d2-b425-d9fc4603fc4b None None] MFA Rules not processed for user `1db83b7bf26f40079752392f6259caf8`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:36:06.676115 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-7521e5b3-1e9b-48d2-b425-d9fc4603fc4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:36:06.741466 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1141/2270] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:36:06 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:36:06.752997 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff05174-592f-4ab3-8ac5-3759dcb3dcb6 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:36:06.754734 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-4ff05174-592f-4ab3-8ac5-3759dcb3dcb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:36:06.813700 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff05174-592f-4ab3-8ac5-3759dcb3dcb6 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:36:06.817392 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b5eb28-b985-40ae-ae7d-1bdf3091a813 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1be1e6173e374a60b9e1fbf51a1ab571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:36:06.817997 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b5eb28-b985-40ae-ae7d-1bdf3091a813 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:36:06.818275 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b5eb28-b985-40ae-ae7d-1bdf3091a813 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:36:06.818548 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b5eb28-b985-40ae-ae7d-1bdf3091a813 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:36:06.827742 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b5eb28-b985-40ae-ae7d-1bdf3091a813 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:36:06.829285 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b5eb28-b985-40ae-ae7d-1bdf3091a813 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:36:06.838611 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1130/2271] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:36:06 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:36:06.858794 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7521e5b3-1e9b-48d2-b425-d9fc4603fc4b None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:36:06.863120 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:36:06.926224 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7521e5b3-1e9b-48d2-b425-d9fc4603fc4b None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:36:06.930803 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262370fb-10e3-4263-815a-b859d2dfa33e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:36:06.931438 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262370fb-10e3-4263-815a-b859d2dfa33e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:36:06.931718 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262370fb-10e3-4263-815a-b859d2dfa33e admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:36:06.931990 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262370fb-10e3-4263-815a-b859d2dfa33e admin admin] PATH_INFO: `/v3/users/1db83b7bf26f40079752392f6259caf8` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:36:06.933209 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262370fb-10e3-4263-815a-b859d2dfa33e admin admin] RBAC: Authorizing `identity:delete_user(user_id=1db83b7bf26f40079752392f6259caf8)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:36:06.935706 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262370fb-10e3-4263-815a-b859d2dfa33e admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:36:06.981646 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-262370fb-10e3-4263-815a-b859d2dfa33e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1db83b7bf26f40079752392f6259caf8'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:36:06.989396 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-262370fb-10e3-4263-815a-b859d2dfa33e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1db83b7bf26f40079752392f6259caf8'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:36:06.990298 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-262370fb-10e3-4263-815a-b859d2dfa33e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1db83b7bf26f40079752392f6259caf8'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:36:06.991067 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-262370fb-10e3-4263-815a-b859d2dfa33e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1db83b7bf26f40079752392f6259caf8'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:36:07.004746 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-262370fb-10e3-4263-815a-b859d2dfa33e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1db83b7bf26f40079752392f6259caf8'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:36:07.018420 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1142/2272] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Aug 6 01:36:06 2022] DELETE /identity/v3/users/1db83b7bf26f40079752392f6259caf8 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:36:07.023020 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29fa34e-b4df-4397-b9d7-b6df8054bf5a None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:36:07.023312 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29fa34e-b4df-4397-b9d7-b6df8054bf5a None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:36:07.023948 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29fa34e-b4df-4397-b9d7-b6df8054bf5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:36:07.042809 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-a29fa34e-b4df-4397-b9d7-b6df8054bf5a None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:36:07.044648 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-a29fa34e-b4df-4397-b9d7-b6df8054bf5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:36:07.097652 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:36:07.108270 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1131/2273] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:36:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:36:07.117843 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262370fb-10e3-4263-815a-b859d2dfa33e admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:36:07.119239 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-262370fb-10e3-4263-815a-b859d2dfa33e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:36:07.177210 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262370fb-10e3-4263-815a-b859d2dfa33e admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:36:07.180762 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83087cbe-cf4a-48bb-b859-c90822648397 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:36:07.181477 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83087cbe-cf4a-48bb-b859-c90822648397 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:36:07.181755 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83087cbe-cf4a-48bb-b859-c90822648397 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:36:07.182023 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83087cbe-cf4a-48bb-b859-c90822648397 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:36:07.191735 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83087cbe-cf4a-48bb-b859-c90822648397 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:36:07.193883 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83087cbe-cf4a-48bb-b859-c90822648397 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:36:07.205307 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1143/2274] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:36:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:36:07.469120 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29fa34e-b4df-4397-b9d7-b6df8054bf5a None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:36:07.470560 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-a29fa34e-b4df-4397-b9d7-b6df8054bf5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:36:07.485608 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29fa34e-b4df-4397-b9d7-b6df8054bf5a None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:36:07.491427 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d4ebfd-69a5-4f21-87ff-aa71a36cb023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:36:07.492153 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d4ebfd-69a5-4f21-87ff-aa71a36cb023 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:36:07.492335 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d4ebfd-69a5-4f21-87ff-aa71a36cb023 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:36:07.492609 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d4ebfd-69a5-4f21-87ff-aa71a36cb023 admin admin] PATH_INFO: `/v3/projects/e38b52a908c9435e843fde297df84b8b` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:36:07.493787 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d4ebfd-69a5-4f21-87ff-aa71a36cb023 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e38b52a908c9435e843fde297df84b8b)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:36:07.495700 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d4ebfd-69a5-4f21-87ff-aa71a36cb023 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:36:07.545213 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-11d4ebfd-69a5-4f21-87ff-aa71a36cb023 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e38b52a908c9435e843fde297df84b8b', 'request_id': 'req-11d4ebfd-69a5-4f21-87ff-aa71a36cb023'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:36:07.551357 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-11d4ebfd-69a5-4f21-87ff-aa71a36cb023 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e38b52a908c9435e843fde297df84b8b', 'request_id': 'req-11d4ebfd-69a5-4f21-87ff-aa71a36cb023'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:36:07.556830 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-11d4ebfd-69a5-4f21-87ff-aa71a36cb023 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e38b52a908c9435e843fde297df84b8b', 'request_id': 'req-11d4ebfd-69a5-4f21-87ff-aa71a36cb023'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:36:07.568971 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-11d4ebfd-69a5-4f21-87ff-aa71a36cb023 admin admin] The token cache is being invalidate because project e38b52a908c9435e843fde297df84b8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:36:07.569289 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-11d4ebfd-69a5-4f21-87ff-aa71a36cb023 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:36:07.569909 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-11d4ebfd-69a5-4f21-87ff-aa71a36cb023 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:36:07.571403 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1132/2275] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Aug 6 01:36:07 2022] DELETE /identity/v3/projects/e38b52a908c9435e843fde297df84b8b => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:36:07.576997 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3427423-bce4-4305-a8ac-591302a2a3f7 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:36:07.577547 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3427423-bce4-4305-a8ac-591302a2a3f7 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:36:07.577830 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3427423-bce4-4305-a8ac-591302a2a3f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:36:07.597797 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-b3427423-bce4-4305-a8ac-591302a2a3f7 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:36:07.598995 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-b3427423-bce4-4305-a8ac-591302a2a3f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:36:07.617777 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:36:07.659646 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1144/2276] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:36:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:36:07.667478 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75e0978-97e1-4874-bf2a-786eef5e972e None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:36:07.667804 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75e0978-97e1-4874-bf2a-786eef5e972e None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:36:07.668085 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75e0978-97e1-4874-bf2a-786eef5e972e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:36:07.687711 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-f75e0978-97e1-4874-bf2a-786eef5e972e None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:36:07.688918 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-f75e0978-97e1-4874-bf2a-786eef5e972e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:36:07.696747 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1133/2277] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:36:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:36:07.701748 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3427423-bce4-4305-a8ac-591302a2a3f7 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:36:07.711370 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3427423-bce4-4305-a8ac-591302a2a3f7 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:36:07.716561 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc476e5-f4a9-4c9e-b651-6d6cc0c260a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:36:07.717202 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc476e5-f4a9-4c9e-b651-6d6cc0c260a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:36:07.717484 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc476e5-f4a9-4c9e-b651-6d6cc0c260a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:36:07.717755 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc476e5-f4a9-4c9e-b651-6d6cc0c260a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:36:07.718453 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc476e5-f4a9-4c9e-b651-6d6cc0c260a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:36:07.718773 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc476e5-f4a9-4c9e-b651-6d6cc0c260a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:36:07.720316 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc476e5-f4a9-4c9e-b651-6d6cc0c260a5 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:36:07.732889 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1145/2278] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Aug 6 01:36:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:36:07.736980 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75e0978-97e1-4874-bf2a-786eef5e972e None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:36:07.746993 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75e0978-97e1-4874-bf2a-786eef5e972e None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:36:07.751926 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a73dde-cbf4-4be2-86e2-75026a83b90d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:36:07.752513 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a73dde-cbf4-4be2-86e2-75026a83b90d admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:36:07.752791 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a73dde-cbf4-4be2-86e2-75026a83b90d admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:36:07.753088 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a73dde-cbf4-4be2-86e2-75026a83b90d admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:36:07.753761 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a73dde-cbf4-4be2-86e2-75026a83b90d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:36:07.755639 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a73dde-cbf4-4be2-86e2-75026a83b90d admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:36:07.784043 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1134/2279] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Aug 6 01:36:07 2022] POST /identity/v3/projects => generated 377 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:36:07.788231 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc476e5-f4a9-4c9e-b651-6d6cc0c260a5 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:36:07.798425 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc476e5-f4a9-4c9e-b651-6d6cc0c260a5 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:36:07.855418 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7151c261-3c82-4c6f-889e-650ef1caefe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:36:07.856149 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7151c261-3c82-4c6f-889e-650ef1caefe7 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:36:07.856433 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7151c261-3c82-4c6f-889e-650ef1caefe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:36:07.856707 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7151c261-3c82-4c6f-889e-650ef1caefe7 admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:36:07.857402 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7151c261-3c82-4c6f-889e-650ef1caefe7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:36:07.859338 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7151c261-3c82-4c6f-889e-650ef1caefe7 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:36:07.881146 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1146/2280] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:36:07 2022] POST /identity/v3/users => generated 289 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:36:07.885302 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a73dde-cbf4-4be2-86e2-75026a83b90d admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:36:07.896073 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a73dde-cbf4-4be2-86e2-75026a83b90d admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:36:07.900688 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a2691d-d085-46a3-bcc4-fe6fff81f534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:36:07.901329 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a2691d-d085-46a3-bcc4-fe6fff81f534 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:36:07.901615 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a2691d-d085-46a3-bcc4-fe6fff81f534 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:36:07.901892 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a2691d-d085-46a3-bcc4-fe6fff81f534 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:36:07.902647 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a2691d-d085-46a3-bcc4-fe6fff81f534 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:36:07.904245 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a2691d-d085-46a3-bcc4-fe6fff81f534 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:36:07.914079 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1135/2281] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:36:07 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:36:07.917967 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7151c261-3c82-4c6f-889e-650ef1caefe7 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:36:07.928655 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7151c261-3c82-4c6f-889e-650ef1caefe7 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:36:07.933473 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285c742d-1d93-4d3f-9d5d-a5e4b73f1d62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:36:07.934797 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285c742d-1d93-4d3f-9d5d-a5e4b73f1d62 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:36:07.935081 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285c742d-1d93-4d3f-9d5d-a5e4b73f1d62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:36:07.935349 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285c742d-1d93-4d3f-9d5d-a5e4b73f1d62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:36:07.936047 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285c742d-1d93-4d3f-9d5d-a5e4b73f1d62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:36:07.937695 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285c742d-1d93-4d3f-9d5d-a5e4b73f1d62 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:36:07.948805 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1147/2282] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:36:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:36:07.953772 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a2691d-d085-46a3-bcc4-fe6fff81f534 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:36:07.964010 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a2691d-d085-46a3-bcc4-fe6fff81f534 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:36:07.969406 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4cf7b5-3617-44f9-958b-719f36f91b66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:36:07.970068 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4cf7b5-3617-44f9-958b-719f36f91b66 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:36:07.970348 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4cf7b5-3617-44f9-958b-719f36f91b66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:36:07.970620 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4cf7b5-3617-44f9-958b-719f36f91b66 admin admin] PATH_INFO: `/v3/projects/f5216aba3b3940528aab64068d4ffdd7/users/6cabeddcfe31489883f558af5b6b7705/roles/61b3b93d37a047e4bb8369b46fe20db9` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:36:07.986687 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4cf7b5-3617-44f9-958b-719f36f91b66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5216aba3b3940528aab64068d4ffdd7, user_id=6cabeddcfe31489883f558af5b6b7705, role_id=61b3b93d37a047e4bb8369b46fe20db9)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:36:07.989384 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4cf7b5-3617-44f9-958b-719f36f91b66 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:36:08.017282 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1136/2283] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:36:07 2022] PUT /identity/v3/projects/f5216aba3b3940528aab64068d4ffdd7/users/6cabeddcfe31489883f558af5b6b7705/roles/61b3b93d37a047e4bb8369b46fe20db9 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:36:08.027265 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c11e2c5-1cac-4c05-b715-db13d3307da4 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:36:08.028463 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c11e2c5-1cac-4c05-b715-db13d3307da4 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:36:08.028748 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c11e2c5-1cac-4c05-b715-db13d3307da4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:36:08.061618 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-7c11e2c5-1cac-4c05-b715-db13d3307da4 None None] MFA Rules not processed for user `6cabeddcfe31489883f558af5b6b7705`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:36:08.062837 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-7c11e2c5-1cac-4c05-b715-db13d3307da4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:36:08.132650 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1148/2284] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:36:08 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:36:08.150463 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4cf7b5-3617-44f9-958b-719f36f91b66 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:36:08.152349 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-0e4cf7b5-3617-44f9-958b-719f36f91b66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:36:08.212464 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4cf7b5-3617-44f9-958b-719f36f91b66 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:36:08.216223 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b0eac5-40ee-44f9-a247-3c8e40fa3cb4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1be1e6173e374a60b9e1fbf51a1ab571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:36:08.216772 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b0eac5-40ee-44f9-a247-3c8e40fa3cb4 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:36:08.217071 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b0eac5-40ee-44f9-a247-3c8e40fa3cb4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:36:08.217347 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b0eac5-40ee-44f9-a247-3c8e40fa3cb4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:36:08.225486 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b0eac5-40ee-44f9-a247-3c8e40fa3cb4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:36:08.226920 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b0eac5-40ee-44f9-a247-3c8e40fa3cb4 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:36:08.236494 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1137/2285] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:36:08 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:36:18.129939 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=88,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:36:18.232654 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=80,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:36:41.998921 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c11e2c5-1cac-4c05-b715-db13d3307da4 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:36:42.001155 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-7c11e2c5-1cac-4c05-b715-db13d3307da4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:36:42.082290 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c11e2c5-1cac-4c05-b715-db13d3307da4 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:36:42.088473 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050d11a1-28c9-42a8-b1c7-3c868958deaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:36:42.089271 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050d11a1-28c9-42a8-b1c7-3c868958deaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:36:42.089606 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050d11a1-28c9-42a8-b1c7-3c868958deaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:36:42.089890 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050d11a1-28c9-42a8-b1c7-3c868958deaa admin admin] PATH_INFO: `/v3/users/b9be4806b0b64d88bdb45733cec065d8` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:36:42.091429 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050d11a1-28c9-42a8-b1c7-3c868958deaa admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9be4806b0b64d88bdb45733cec065d8)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:36:42.095166 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050d11a1-28c9-42a8-b1c7-3c868958deaa admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:36:42.155460 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-050d11a1-28c9-42a8-b1c7-3c868958deaa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9be4806b0b64d88bdb45733cec065d8'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:36:42.166528 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-050d11a1-28c9-42a8-b1c7-3c868958deaa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9be4806b0b64d88bdb45733cec065d8'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:36:42.167518 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-050d11a1-28c9-42a8-b1c7-3c868958deaa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9be4806b0b64d88bdb45733cec065d8'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:36:42.168989 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-050d11a1-28c9-42a8-b1c7-3c868958deaa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9be4806b0b64d88bdb45733cec065d8'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:36:42.185884 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-050d11a1-28c9-42a8-b1c7-3c868958deaa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9be4806b0b64d88bdb45733cec065d8'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:36:42.211522 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1149/2286] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Aug 6 01:36:41 2022] DELETE /identity/v3/users/b9be4806b0b64d88bdb45733cec065d8 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:36:42.219635 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12786ced-8dd5-43cd-af6c-59a0a458c629 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:36:42.219635 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12786ced-8dd5-43cd-af6c-59a0a458c629 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:36:42.219635 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12786ced-8dd5-43cd-af6c-59a0a458c629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:36:42.260481 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-12786ced-8dd5-43cd-af6c-59a0a458c629 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:36:42.261878 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-12786ced-8dd5-43cd-af6c-59a0a458c629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:36:42.335402 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1138/2287] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:36:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:36:42.344164 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050d11a1-28c9-42a8-b1c7-3c868958deaa admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:36:42.345855 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-050d11a1-28c9-42a8-b1c7-3c868958deaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:36:42.418596 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050d11a1-28c9-42a8-b1c7-3c868958deaa admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:36:42.422613 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce416858-cd45-4455-8264-c47be14c6929 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:36:42.423308 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce416858-cd45-4455-8264-c47be14c6929 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:36:42.423658 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce416858-cd45-4455-8264-c47be14c6929 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:36:42.424022 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce416858-cd45-4455-8264-c47be14c6929 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:36:42.435169 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce416858-cd45-4455-8264-c47be14c6929 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:36:42.436763 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce416858-cd45-4455-8264-c47be14c6929 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:36:42.437526 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12786ced-8dd5-43cd-af6c-59a0a458c629 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:36:42.448744 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12786ced-8dd5-43cd-af6c-59a0a458c629 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:36:42.449402 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1150/2288] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:36:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:36:42.457392 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0c02fc-b642-4c07-89a0-1ec5f9fc5bc3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:36:42.458162 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0c02fc-b642-4c07-89a0-1ec5f9fc5bc3 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:36:42.458518 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0c02fc-b642-4c07-89a0-1ec5f9fc5bc3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:36:42.458858 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0c02fc-b642-4c07-89a0-1ec5f9fc5bc3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:36:42.460923 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-dc0c02fc-b642-4c07-89a0-1ec5f9fc5bc3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:36:42.570316 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0c02fc-b642-4c07-89a0-1ec5f9fc5bc3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:36:42.571971 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0c02fc-b642-4c07-89a0-1ec5f9fc5bc3 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:36:42.588493 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1139/2289] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:36:42 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 01:36:42.895521 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce416858-cd45-4455-8264-c47be14c6929 service neutron] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:36:42.897027 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-ce416858-cd45-4455-8264-c47be14c6929 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:36:42.928387 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce416858-cd45-4455-8264-c47be14c6929 service neutron] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:36:42.933923 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c6471c-94c0-43bd-9a91-67d2b355637e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:36:42.934859 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c6471c-94c0-43bd-9a91-67d2b355637e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:36:42.935282 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c6471c-94c0-43bd-9a91-67d2b355637e admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:36:42.935704 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c6471c-94c0-43bd-9a91-67d2b355637e admin admin] PATH_INFO: `/v3/projects/1e7cfc49aa79444f8681a8be334a40bd` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:36:42.938723 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c6471c-94c0-43bd-9a91-67d2b355637e admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e7cfc49aa79444f8681a8be334a40bd)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:36:42.941418 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c6471c-94c0-43bd-9a91-67d2b355637e admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:36:42.965922 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:36:43.026644 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-71c6471c-94c0-43bd-9a91-67d2b355637e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e7cfc49aa79444f8681a8be334a40bd', 'request_id': 'req-71c6471c-94c0-43bd-9a91-67d2b355637e'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:36:43.036117 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-71c6471c-94c0-43bd-9a91-67d2b355637e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e7cfc49aa79444f8681a8be334a40bd', 'request_id': 'req-71c6471c-94c0-43bd-9a91-67d2b355637e'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:36:43.045342 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-71c6471c-94c0-43bd-9a91-67d2b355637e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e7cfc49aa79444f8681a8be334a40bd', 'request_id': 'req-71c6471c-94c0-43bd-9a91-67d2b355637e'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:36:43.066139 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-71c6471c-94c0-43bd-9a91-67d2b355637e admin admin] The token cache is being invalidate because project 1e7cfc49aa79444f8681a8be334a40bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:36:43.066453 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-71c6471c-94c0-43bd-9a91-67d2b355637e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:36:43.067632 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-71c6471c-94c0-43bd-9a91-67d2b355637e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:36:43.069686 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1151/2290] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Aug 6 01:36:42 2022] DELETE /identity/v3/projects/1e7cfc49aa79444f8681a8be334a40bd => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:36:43.075465 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1250f915-09e9-47d9-99a7-c3d8eaec8327 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:36:43.075929 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1250f915-09e9-47d9-99a7-c3d8eaec8327 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:36:43.076697 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1250f915-09e9-47d9-99a7-c3d8eaec8327 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:36:43.102603 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-1250f915-09e9-47d9-99a7-c3d8eaec8327 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:36:43.103971 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-1250f915-09e9-47d9-99a7-c3d8eaec8327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:36:43.223987 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1140/2291] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:36:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:36:43.227094 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57982001-5ef4-48ca-8aa1-92e7ad6a2077 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:36:43.227384 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57982001-5ef4-48ca-8aa1-92e7ad6a2077 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:36:43.227680 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57982001-5ef4-48ca-8aa1-92e7ad6a2077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:36:43.258587 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-57982001-5ef4-48ca-8aa1-92e7ad6a2077 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:36:43.260364 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-57982001-5ef4-48ca-8aa1-92e7ad6a2077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:36:43.274290 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1152/2292] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:36:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:36:43.275054 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1250f915-09e9-47d9-99a7-c3d8eaec8327 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:36:43.286707 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1250f915-09e9-47d9-99a7-c3d8eaec8327 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:36:43.292326 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ab3c56-8dc6-44cc-a9d1-fd521b025d70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:36:43.294139 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ab3c56-8dc6-44cc-a9d1-fd521b025d70 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:36:43.294139 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ab3c56-8dc6-44cc-a9d1-fd521b025d70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:36:43.294139 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ab3c56-8dc6-44cc-a9d1-fd521b025d70 admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:36:43.294598 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ab3c56-8dc6-44cc-a9d1-fd521b025d70 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:36:43.295183 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ab3c56-8dc6-44cc-a9d1-fd521b025d70 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:36:43.296899 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ab3c56-8dc6-44cc-a9d1-fd521b025d70 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:36:43.313425 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1141/2293] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Aug 6 01:36:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:36:43.319185 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57982001-5ef4-48ca-8aa1-92e7ad6a2077 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:36:43.329099 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57982001-5ef4-48ca-8aa1-92e7ad6a2077 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:36:43.334454 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57629327-c594-4f69-a91f-905ac94491d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:36:43.335068 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57629327-c594-4f69-a91f-905ac94491d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:36:43.335372 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57629327-c594-4f69-a91f-905ac94491d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:36:43.335643 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57629327-c594-4f69-a91f-905ac94491d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:36:43.336362 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57629327-c594-4f69-a91f-905ac94491d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:36:43.338264 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57629327-c594-4f69-a91f-905ac94491d5 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:36:43.372685 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1153/2294] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Aug 6 01:36:43 2022] POST /identity/v3/projects => generated 375 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:36:43.377496 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ab3c56-8dc6-44cc-a9d1-fd521b025d70 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:36:43.388190 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ab3c56-8dc6-44cc-a9d1-fd521b025d70 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:36:43.412310 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:36:43.453813 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b90b2a4-1134-40b8-a5d6-7e530111cd4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:36:43.454605 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b90b2a4-1134-40b8-a5d6-7e530111cd4a admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:36:43.454899 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b90b2a4-1134-40b8-a5d6-7e530111cd4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:36:43.455251 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b90b2a4-1134-40b8-a5d6-7e530111cd4a admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:36:43.455957 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b90b2a4-1134-40b8-a5d6-7e530111cd4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:36:43.457945 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b90b2a4-1134-40b8-a5d6-7e530111cd4a admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:36:43.482484 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1142/2295] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:36:43 2022] POST /identity/v3/users => generated 288 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:36:43.487309 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57629327-c594-4f69-a91f-905ac94491d5 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:36:43.498779 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57629327-c594-4f69-a91f-905ac94491d5 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:36:43.504909 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d58879-b16a-4ffa-9c78-192e2755218d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:36:43.505754 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d58879-b16a-4ffa-9c78-192e2755218d admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:36:43.506090 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d58879-b16a-4ffa-9c78-192e2755218d admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:36:43.506375 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d58879-b16a-4ffa-9c78-192e2755218d admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:36:43.507411 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d58879-b16a-4ffa-9c78-192e2755218d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:36:43.509300 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d58879-b16a-4ffa-9c78-192e2755218d admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:36:43.522030 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1154/2296] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:36:43 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:36:43.526085 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b90b2a4-1134-40b8-a5d6-7e530111cd4a admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:36:43.536823 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b90b2a4-1134-40b8-a5d6-7e530111cd4a admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:36:43.542260 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea6e9ae-b738-446e-b6f1-7e21a0a6b536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:36:43.542931 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea6e9ae-b738-446e-b6f1-7e21a0a6b536 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:36:43.543245 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea6e9ae-b738-446e-b6f1-7e21a0a6b536 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:36:43.543527 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea6e9ae-b738-446e-b6f1-7e21a0a6b536 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:36:43.544233 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea6e9ae-b738-446e-b6f1-7e21a0a6b536 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:36:43.545888 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea6e9ae-b738-446e-b6f1-7e21a0a6b536 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:36:43.559343 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1143/2297] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:36:43 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:36:43.564384 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d58879-b16a-4ffa-9c78-192e2755218d admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:36:43.574908 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d58879-b16a-4ffa-9c78-192e2755218d admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:36:43.580957 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a939142c-536a-41df-8263-39ff2b8bbea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:36:43.581691 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a939142c-536a-41df-8263-39ff2b8bbea5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:36:43.581971 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a939142c-536a-41df-8263-39ff2b8bbea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:36:43.582255 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a939142c-536a-41df-8263-39ff2b8bbea5 admin admin] PATH_INFO: `/v3/projects/c56b8479508d4ca994d5f3035c8d18e8/users/5471e0d1dbfd4858ab89950cc52c639b/roles/61b3b93d37a047e4bb8369b46fe20db9` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:36:43.600178 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a939142c-536a-41df-8263-39ff2b8bbea5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c56b8479508d4ca994d5f3035c8d18e8, user_id=5471e0d1dbfd4858ab89950cc52c639b, role_id=61b3b93d37a047e4bb8369b46fe20db9)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:36:43.603733 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a939142c-536a-41df-8263-39ff2b8bbea5 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:36:43.629823 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1155/2298] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:36:43 2022] PUT /identity/v3/projects/c56b8479508d4ca994d5f3035c8d18e8/users/5471e0d1dbfd4858ab89950cc52c639b/roles/61b3b93d37a047e4bb8369b46fe20db9 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:36:43.632391 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cfd8e6-b937-4061-8d79-07fe773b8586 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:36:43.632685 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cfd8e6-b937-4061-8d79-07fe773b8586 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:36:43.632961 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cfd8e6-b937-4061-8d79-07fe773b8586 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:36:43.663892 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-33cfd8e6-b937-4061-8d79-07fe773b8586 None None] MFA Rules not processed for user `5471e0d1dbfd4858ab89950cc52c639b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:36:43.665175 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-33cfd8e6-b937-4061-8d79-07fe773b8586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:36:43.761069 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1144/2299] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:36:43 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:36:43.771575 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a939142c-536a-41df-8263-39ff2b8bbea5 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:36:43.773697 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-a939142c-536a-41df-8263-39ff2b8bbea5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:36:43.851586 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a939142c-536a-41df-8263-39ff2b8bbea5 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:36:43.858206 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937a150a-db57-4544-984b-0ef894bf8826 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1be1e6173e374a60b9e1fbf51a1ab571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:36:43.858966 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937a150a-db57-4544-984b-0ef894bf8826 service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:36:43.859359 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937a150a-db57-4544-984b-0ef894bf8826 service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:36:43.859835 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937a150a-db57-4544-984b-0ef894bf8826 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:36:43.872870 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937a150a-db57-4544-984b-0ef894bf8826 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:36:43.872870 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937a150a-db57-4544-984b-0ef894bf8826 service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:36:43.897194 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1156/2300] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:36:43 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:36:50.251254 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cfd8e6-b937-4061-8d79-07fe773b8586 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:36:50.252716 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-33cfd8e6-b937-4061-8d79-07fe773b8586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:36:50.332552 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cfd8e6-b937-4061-8d79-07fe773b8586 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:36:50.337873 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29973750-3645-4d57-82a8-d5e85efd5a55 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa3b93ddd9464a2cba724d67b56b477a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:36:50.338499 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29973750-3645-4d57-82a8-d5e85efd5a55 service placement] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:36:50.338785 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29973750-3645-4d57-82a8-d5e85efd5a55 service placement] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:36:50.339065 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29973750-3645-4d57-82a8-d5e85efd5a55 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:36:50.340900 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-29973750-3645-4d57-82a8-d5e85efd5a55 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:36:50.353755 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29973750-3645-4d57-82a8-d5e85efd5a55 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:36:50.355520 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29973750-3645-4d57-82a8-d5e85efd5a55 service placement] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:36:50.366845 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1145/2301] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:36:50 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:36:50.530522 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c3cb52-eefa-463d-bede-607e2d8e7682 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:36:50.531074 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c3cb52-eefa-463d-bede-607e2d8e7682 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:36:50.531074 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c3cb52-eefa-463d-bede-607e2d8e7682 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:36:50.549955 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-24c3cb52-eefa-463d-bede-607e2d8e7682 None None] MFA Rules not processed for user `5471e0d1dbfd4858ab89950cc52c639b`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:36:50.551152 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-24c3cb52-eefa-463d-bede-607e2d8e7682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:36:50.557961 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1157/2302] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:36:50 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:36:59.358573 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29973750-3645-4d57-82a8-d5e85efd5a55 service placement] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:36:59.360311 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-29973750-3645-4d57-82a8-d5e85efd5a55 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:36:59.386091 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:36:59.443702 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29973750-3645-4d57-82a8-d5e85efd5a55 service placement] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:36:59.449105 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f55fcb-bcf5-4fb5-8db8-d97fabaf3683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:36:59.449874 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f55fcb-bcf5-4fb5-8db8-d97fabaf3683 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:36:59.450266 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f55fcb-bcf5-4fb5-8db8-d97fabaf3683 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:36:59.450661 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f55fcb-bcf5-4fb5-8db8-d97fabaf3683 admin admin] PATH_INFO: `/v3/users/6cabeddcfe31489883f558af5b6b7705` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:36:59.451965 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f55fcb-bcf5-4fb5-8db8-d97fabaf3683 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6cabeddcfe31489883f558af5b6b7705)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:36:59.454320 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f55fcb-bcf5-4fb5-8db8-d97fabaf3683 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:36:59.508836 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-74f55fcb-bcf5-4fb5-8db8-d97fabaf3683 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6cabeddcfe31489883f558af5b6b7705'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:36:59.509559 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-74f55fcb-bcf5-4fb5-8db8-d97fabaf3683 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6cabeddcfe31489883f558af5b6b7705'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:36:59.510222 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-74f55fcb-bcf5-4fb5-8db8-d97fabaf3683 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6cabeddcfe31489883f558af5b6b7705'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:36:59.522339 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-74f55fcb-bcf5-4fb5-8db8-d97fabaf3683 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6cabeddcfe31489883f558af5b6b7705'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:36:59.529018 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-74f55fcb-bcf5-4fb5-8db8-d97fabaf3683 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6cabeddcfe31489883f558af5b6b7705'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:36:59.542206 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1146/2303] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Aug 6 01:36:59 2022] DELETE /identity/v3/users/6cabeddcfe31489883f558af5b6b7705 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:36:59.547690 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b05ad74-6c39-4bee-b0f9-1dbd8462e889 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:36:59.548835 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b05ad74-6c39-4bee-b0f9-1dbd8462e889 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:36:59.548966 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b05ad74-6c39-4bee-b0f9-1dbd8462e889 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:36:59.569402 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-0b05ad74-6c39-4bee-b0f9-1dbd8462e889 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:36:59.570625 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-0b05ad74-6c39-4bee-b0f9-1dbd8462e889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:36:59.602911 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:36:59.636650 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1158/2304] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:36:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:36:59.646950 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f55fcb-bcf5-4fb5-8db8-d97fabaf3683 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:36:59.648386 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-74f55fcb-bcf5-4fb5-8db8-d97fabaf3683 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:36:59.707345 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f55fcb-bcf5-4fb5-8db8-d97fabaf3683 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:36:59.711060 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682f02a1-2787-4cc6-b60a-954909d9a5d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:36:59.711684 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682f02a1-2787-4cc6-b60a-954909d9a5d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:36:59.711966 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682f02a1-2787-4cc6-b60a-954909d9a5d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:36:59.712244 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682f02a1-2787-4cc6-b60a-954909d9a5d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:36:59.721552 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682f02a1-2787-4cc6-b60a-954909d9a5d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:36:59.722972 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682f02a1-2787-4cc6-b60a-954909d9a5d2 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:36:59.732330 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1147/2305] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:36:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:37:00.005248 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b05ad74-6c39-4bee-b0f9-1dbd8462e889 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:37:00.006963 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-0b05ad74-6c39-4bee-b0f9-1dbd8462e889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:37:00.019789 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b05ad74-6c39-4bee-b0f9-1dbd8462e889 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:37:00.025918 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68b6ef7-5a97-4f5d-9d41-38adbd83d228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:37:00.026565 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68b6ef7-5a97-4f5d-9d41-38adbd83d228 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:37:00.026972 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68b6ef7-5a97-4f5d-9d41-38adbd83d228 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:37:00.027149 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68b6ef7-5a97-4f5d-9d41-38adbd83d228 admin admin] PATH_INFO: `/v3/projects/f5216aba3b3940528aab64068d4ffdd7` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:37:00.028958 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68b6ef7-5a97-4f5d-9d41-38adbd83d228 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5216aba3b3940528aab64068d4ffdd7)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:37:00.031087 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68b6ef7-5a97-4f5d-9d41-38adbd83d228 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:37:00.081836 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-c68b6ef7-5a97-4f5d-9d41-38adbd83d228 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5216aba3b3940528aab64068d4ffdd7', 'request_id': 'req-c68b6ef7-5a97-4f5d-9d41-38adbd83d228'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:37:00.088207 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-c68b6ef7-5a97-4f5d-9d41-38adbd83d228 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5216aba3b3940528aab64068d4ffdd7', 'request_id': 'req-c68b6ef7-5a97-4f5d-9d41-38adbd83d228'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:37:00.093755 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-c68b6ef7-5a97-4f5d-9d41-38adbd83d228 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5216aba3b3940528aab64068d4ffdd7', 'request_id': 'req-c68b6ef7-5a97-4f5d-9d41-38adbd83d228'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:37:00.105986 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-c68b6ef7-5a97-4f5d-9d41-38adbd83d228 admin admin] The token cache is being invalidate because project f5216aba3b3940528aab64068d4ffdd7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:37:00.106275 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-c68b6ef7-5a97-4f5d-9d41-38adbd83d228 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:37:00.106909 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-c68b6ef7-5a97-4f5d-9d41-38adbd83d228 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:37:00.108411 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1159/2306] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Aug 6 01:37:00 2022] DELETE /identity/v3/projects/f5216aba3b3940528aab64068d4ffdd7 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:37:06.573328 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79805d18-5590-46f1-8868-9b4d0bcbc92a None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:37:06.573817 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79805d18-5590-46f1-8868-9b4d0bcbc92a None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:37:06.573909 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79805d18-5590-46f1-8868-9b4d0bcbc92a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:37:06.599856 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-79805d18-5590-46f1-8868-9b4d0bcbc92a None None] MFA Rules not processed for user `c938a575a2704078a82fe6df941884dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:37:06.601370 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-79805d18-5590-46f1-8868-9b4d0bcbc92a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:37:06.654770 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:37:06.679593 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1148/2307] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:37:06 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:37:10.102900 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=37,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:37:16.671483 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=6 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:37:28.311710 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fd37f5-06a9-4108-8875-99a03a7abdf2 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:37:28.312200 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fd37f5-06a9-4108-8875-99a03a7abdf2 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:37:28.312303 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fd37f5-06a9-4108-8875-99a03a7abdf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:37:28.332774 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-60fd37f5-06a9-4108-8875-99a03a7abdf2 None None] MFA Rules not processed for user `8e86f38590824894a3b960e6774ff8a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:37:28.334081 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-60fd37f5-06a9-4108-8875-99a03a7abdf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:37:28.389617 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1160/2308] 10.222.0.26 () {58 vars in 1009 bytes} [Sat Aug 6 01:37:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:37:28.436060 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79805d18-5590-46f1-8868-9b4d0bcbc92a None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:37:28.437522 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-79805d18-5590-46f1-8868-9b4d0bcbc92a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:37:28.507413 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79805d18-5590-46f1-8868-9b4d0bcbc92a None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:37:28.512400 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f34a2f1-7b9c-479d-9aba-5d488065ba66 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b23f4478f49471fba6f43e6e4be99da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:37:28.513016 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f34a2f1-7b9c-479d-9aba-5d488065ba66 service swift] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:37:28.513325 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f34a2f1-7b9c-479d-9aba-5d488065ba66 service swift] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:37:28.513625 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f34a2f1-7b9c-479d-9aba-5d488065ba66 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:37:28.521604 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f34a2f1-7b9c-479d-9aba-5d488065ba66 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:37:28.523178 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f34a2f1-7b9c-479d-9aba-5d488065ba66 service swift] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:37:28.531178 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1149/2309] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Aug 6 01:37:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 06 01:37:29.081694 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47c88d1-683e-46cd-a274-4472fa3f5482 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:37:29.081985 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47c88d1-683e-46cd-a274-4472fa3f5482 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:37:29.082266 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47c88d1-683e-46cd-a274-4472fa3f5482 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:37:29.102014 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-b47c88d1-683e-46cd-a274-4472fa3f5482 None None] MFA Rules not processed for user `8e86f38590824894a3b960e6774ff8a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:37:29.103194 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-b47c88d1-683e-46cd-a274-4472fa3f5482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:37:29.110231 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1161/2310] 10.222.0.26 () {58 vars in 967 bytes} [Sat Aug 6 01:37:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:37:29.123016 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f34a2f1-7b9c-479d-9aba-5d488065ba66 service swift] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:37:29.134213 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f34a2f1-7b9c-479d-9aba-5d488065ba66 service swift] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:37:29.139425 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfffa82-1fd3-495d-8f91-043a7c6cf961 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b23f4478f49471fba6f43e6e4be99da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:37:29.140040 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfffa82-1fd3-495d-8f91-043a7c6cf961 service swift] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:37:29.140329 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfffa82-1fd3-495d-8f91-043a7c6cf961 service swift] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:37:29.140610 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfffa82-1fd3-495d-8f91-043a7c6cf961 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:37:29.149366 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfffa82-1fd3-495d-8f91-043a7c6cf961 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:37:29.150909 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfffa82-1fd3-495d-8f91-043a7c6cf961 service swift] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:37:29.159305 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1150/2311] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Aug 6 01:37:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 06 01:37:29.395767 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11c8981-e003-48e6-b6e9-47d39ff2262c None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:37:29.396253 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11c8981-e003-48e6-b6e9-47d39ff2262c None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:37:29.396358 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11c8981-e003-48e6-b6e9-47d39ff2262c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:37:29.415818 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-c11c8981-e003-48e6-b6e9-47d39ff2262c None None] MFA Rules not processed for user `8e86f38590824894a3b960e6774ff8a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:37:29.417032 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-c11c8981-e003-48e6-b6e9-47d39ff2262c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:37:29.424405 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1162/2312] 10.222.0.26 () {58 vars in 1009 bytes} [Sat Aug 6 01:37:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:37:29.437310 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfffa82-1fd3-495d-8f91-043a7c6cf961 service swift] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:37:29.447863 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfffa82-1fd3-495d-8f91-043a7c6cf961 service swift] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:37:29.453033 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9361f488-6c8f-4027-abdf-948ea04fe59a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b23f4478f49471fba6f43e6e4be99da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:37:29.453741 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9361f488-6c8f-4027-abdf-948ea04fe59a service swift] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:37:29.454034 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9361f488-6c8f-4027-abdf-948ea04fe59a service swift] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:37:29.454314 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9361f488-6c8f-4027-abdf-948ea04fe59a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:37:29.462338 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9361f488-6c8f-4027-abdf-948ea04fe59a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:37:29.463914 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9361f488-6c8f-4027-abdf-948ea04fe59a service swift] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:37:29.472593 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1151/2313] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Aug 6 01:37:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 06 01:37:29.781337 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d95336a-eb5b-4729-bcaf-1bdfd772f786 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:37:29.781625 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d95336a-eb5b-4729-bcaf-1bdfd772f786 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:37:29.781896 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d95336a-eb5b-4729-bcaf-1bdfd772f786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:37:29.801020 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-7d95336a-eb5b-4729-bcaf-1bdfd772f786 None None] MFA Rules not processed for user `8e86f38590824894a3b960e6774ff8a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:37:29.802221 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-7d95336a-eb5b-4729-bcaf-1bdfd772f786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:37:29.809719 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1163/2314] 10.222.0.26 () {58 vars in 967 bytes} [Sat Aug 6 01:37:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:37:29.821706 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9361f488-6c8f-4027-abdf-948ea04fe59a service swift] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:37:29.832934 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9361f488-6c8f-4027-abdf-948ea04fe59a service swift] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:37:29.838048 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5d2af6-b424-4855-9562-a91efbdfa386 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b23f4478f49471fba6f43e6e4be99da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:37:29.838656 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5d2af6-b424-4855-9562-a91efbdfa386 service swift] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:37:29.838940 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5d2af6-b424-4855-9562-a91efbdfa386 service swift] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:37:29.839218 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5d2af6-b424-4855-9562-a91efbdfa386 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:37:29.848226 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5d2af6-b424-4855-9562-a91efbdfa386 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:37:29.849830 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5d2af6-b424-4855-9562-a91efbdfa386 service swift] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:37:29.858110 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1152/2315] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Aug 6 01:37:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 06 01:37:32.200986 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d95336a-eb5b-4729-bcaf-1bdfd772f786 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:37:32.202455 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-7d95336a-eb5b-4729-bcaf-1bdfd772f786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:37:32.254937 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5d2af6-b424-4855-9562-a91efbdfa386 service swift] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:37:32.256456 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-8c5d2af6-b424-4855-9562-a91efbdfa386 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:37:32.280258 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d95336a-eb5b-4729-bcaf-1bdfd772f786 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:37:32.286180 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62cfd82-3baa-40bf-b2ac-bc8e0c0dacdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:37:32.286904 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62cfd82-3baa-40bf-b2ac-bc8e0c0dacdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:37:32.287254 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62cfd82-3baa-40bf-b2ac-bc8e0c0dacdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:37:32.287613 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62cfd82-3baa-40bf-b2ac-bc8e0c0dacdc admin admin] PATH_INFO: `/v3/users/5471e0d1dbfd4858ab89950cc52c639b` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:37:32.288863 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62cfd82-3baa-40bf-b2ac-bc8e0c0dacdc admin admin] RBAC: Authorizing `identity:delete_user(user_id=5471e0d1dbfd4858ab89950cc52c639b)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:37:32.291032 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62cfd82-3baa-40bf-b2ac-bc8e0c0dacdc admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:37:32.346937 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a62cfd82-3baa-40bf-b2ac-bc8e0c0dacdc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5471e0d1dbfd4858ab89950cc52c639b'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:37:32.347909 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5d2af6-b424-4855-9562-a91efbdfa386 service swift] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:37:32.352675 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310c7c87-ee81-422c-8d4c-13ef5bedbde5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:37:32.353350 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310c7c87-ee81-422c-8d4c-13ef5bedbde5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:37:32.353641 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310c7c87-ee81-422c-8d4c-13ef5bedbde5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:37:32.353920 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310c7c87-ee81-422c-8d4c-13ef5bedbde5 admin admin] PATH_INFO: `/v3/users/7b41c8b1389445e59b0d5f388873ec7f` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:37:32.355129 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310c7c87-ee81-422c-8d4c-13ef5bedbde5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b41c8b1389445e59b0d5f388873ec7f)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:37:32.355489 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a62cfd82-3baa-40bf-b2ac-bc8e0c0dacdc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5471e0d1dbfd4858ab89950cc52c639b'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:37:32.356175 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a62cfd82-3baa-40bf-b2ac-bc8e0c0dacdc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5471e0d1dbfd4858ab89950cc52c639b'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:37:32.356821 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a62cfd82-3baa-40bf-b2ac-bc8e0c0dacdc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5471e0d1dbfd4858ab89950cc52c639b'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:37:32.357087 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310c7c87-ee81-422c-8d4c-13ef5bedbde5 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:37:32.371695 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a62cfd82-3baa-40bf-b2ac-bc8e0c0dacdc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5471e0d1dbfd4858ab89950cc52c639b'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:37:32.387993 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1164/2316] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Aug 6 01:37:32 2022] DELETE /identity/v3/users/5471e0d1dbfd4858ab89950cc52c639b => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:37:32.392518 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4ba9eb-e3cd-4c27-a2be-76dff348528e None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:37:32.392812 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4ba9eb-e3cd-4c27-a2be-76dff348528e None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:37:32.393820 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4ba9eb-e3cd-4c27-a2be-76dff348528e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:37:32.403066 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:37:32.416738 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-7a4ba9eb-e3cd-4c27-a2be-76dff348528e None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:37:32.418279 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-7a4ba9eb-e3cd-4c27-a2be-76dff348528e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:37:32.427190 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-310c7c87-ee81-422c-8d4c-13ef5bedbde5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b41c8b1389445e59b0d5f388873ec7f'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:37:32.427965 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-310c7c87-ee81-422c-8d4c-13ef5bedbde5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b41c8b1389445e59b0d5f388873ec7f'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:37:32.428696 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-310c7c87-ee81-422c-8d4c-13ef5bedbde5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b41c8b1389445e59b0d5f388873ec7f'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:37:32.434056 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:37:32.445118 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-310c7c87-ee81-422c-8d4c-13ef5bedbde5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b41c8b1389445e59b0d5f388873ec7f'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:37:32.452107 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-310c7c87-ee81-422c-8d4c-13ef5bedbde5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b41c8b1389445e59b0d5f388873ec7f'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:37:32.466736 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1153/2317] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Aug 6 01:37:32 2022] DELETE /identity/v3/users/7b41c8b1389445e59b0d5f388873ec7f => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:37:32.471187 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef81ac0-491e-49ce-b215-c0e6b17867a8 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:37:32.471526 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef81ac0-491e-49ce-b215-c0e6b17867a8 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:37:32.471900 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef81ac0-491e-49ce-b215-c0e6b17867a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:37:32.482769 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1165/2318] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:37:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:37:32.492665 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4ba9eb-e3cd-4c27-a2be-76dff348528e None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:37:32.492836 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-7ef81ac0-491e-49ce-b215-c0e6b17867a8 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:37:32.493977 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-7ef81ac0-491e-49ce-b215-c0e6b17867a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:37:32.495011 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-7a4ba9eb-e3cd-4c27-a2be-76dff348528e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:37:32.552153 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4ba9eb-e3cd-4c27-a2be-76dff348528e None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:37:32.554058 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1154/2319] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:37:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:37:32.564065 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef81ac0-491e-49ce-b215-c0e6b17867a8 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:37:32.602112 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c805da0-9790-4095-8e84-ccb6026c29a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:37:32.602737 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c805da0-9790-4095-8e84-ccb6026c29a8 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:37:32.603020 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c805da0-9790-4095-8e84-ccb6026c29a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:37:32.603295 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c805da0-9790-4095-8e84-ccb6026c29a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:37:32.613142 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c805da0-9790-4095-8e84-ccb6026c29a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:37:32.614603 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c805da0-9790-4095-8e84-ccb6026c29a8 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:37:32.619668 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef81ac0-491e-49ce-b215-c0e6b17867a8 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:37:32.623388 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4661d970-207f-482d-9fa2-c6699b2dfc4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:37:32.624031 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4661d970-207f-482d-9fa2-c6699b2dfc4f service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:37:32.624315 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4661d970-207f-482d-9fa2-c6699b2dfc4f service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:37:32.624598 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4661d970-207f-482d-9fa2-c6699b2dfc4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:37:32.624729 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1166/2320] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:37:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 01:37:32.634920 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4661d970-207f-482d-9fa2-c6699b2dfc4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:37:32.636451 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4661d970-207f-482d-9fa2-c6699b2dfc4f service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:37:32.647388 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1155/2321] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:37:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 01:37:32.821145 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c805da0-9790-4095-8e84-ccb6026c29a8 service neutron] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:37:32.822942 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-6c805da0-9790-4095-8e84-ccb6026c29a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:37:32.844107 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c805da0-9790-4095-8e84-ccb6026c29a8 service neutron] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:37:32.850490 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9690992-72d2-44cc-a2f6-0efa21466520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:37:32.851391 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9690992-72d2-44cc-a2f6-0efa21466520 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:37:32.851673 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9690992-72d2-44cc-a2f6-0efa21466520 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:37:32.851947 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9690992-72d2-44cc-a2f6-0efa21466520 admin admin] PATH_INFO: `/v3/projects/c56b8479508d4ca994d5f3035c8d18e8` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:37:32.853258 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9690992-72d2-44cc-a2f6-0efa21466520 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c56b8479508d4ca994d5f3035c8d18e8)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:37:32.855160 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9690992-72d2-44cc-a2f6-0efa21466520 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:37:32.917483 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-c9690992-72d2-44cc-a2f6-0efa21466520 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c56b8479508d4ca994d5f3035c8d18e8', 'request_id': 'req-c9690992-72d2-44cc-a2f6-0efa21466520'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:37:32.924738 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-c9690992-72d2-44cc-a2f6-0efa21466520 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c56b8479508d4ca994d5f3035c8d18e8', 'request_id': 'req-c9690992-72d2-44cc-a2f6-0efa21466520'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:37:32.933033 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:37:32.935693 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-c9690992-72d2-44cc-a2f6-0efa21466520 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c56b8479508d4ca994d5f3035c8d18e8', 'request_id': 'req-c9690992-72d2-44cc-a2f6-0efa21466520'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:37:32.954106 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-c9690992-72d2-44cc-a2f6-0efa21466520 admin admin] The token cache is being invalidate because project c56b8479508d4ca994d5f3035c8d18e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:37:32.954434 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-c9690992-72d2-44cc-a2f6-0efa21466520 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:37:32.955112 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-c9690992-72d2-44cc-a2f6-0efa21466520 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:37:32.959731 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1167/2322] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Aug 6 01:37:32 2022] DELETE /identity/v3/projects/c56b8479508d4ca994d5f3035c8d18e8 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:37:33.042459 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4661d970-207f-482d-9fa2-c6699b2dfc4f service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:37:33.044525 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-4661d970-207f-482d-9fa2-c6699b2dfc4f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:37:33.066158 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4661d970-207f-482d-9fa2-c6699b2dfc4f service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:37:33.142867 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8943c10f-b95c-44c3-a259-6854b1cfde09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:37:33.143650 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8943c10f-b95c-44c3-a259-6854b1cfde09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:37:33.143944 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8943c10f-b95c-44c3-a259-6854b1cfde09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:37:33.144228 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8943c10f-b95c-44c3-a259-6854b1cfde09 admin admin] PATH_INFO: `/v3/projects/86651f48c1f441f1b0e115e57b840a28` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:37:33.145511 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8943c10f-b95c-44c3-a259-6854b1cfde09 admin admin] RBAC: Authorizing `identity:delete_project(project_id=86651f48c1f441f1b0e115e57b840a28)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:37:33.147503 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8943c10f-b95c-44c3-a259-6854b1cfde09 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:37:33.166381 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:37:33.197694 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-8943c10f-b95c-44c3-a259-6854b1cfde09 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86651f48c1f441f1b0e115e57b840a28', 'request_id': 'req-8943c10f-b95c-44c3-a259-6854b1cfde09'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:37:33.203735 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-8943c10f-b95c-44c3-a259-6854b1cfde09 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86651f48c1f441f1b0e115e57b840a28', 'request_id': 'req-8943c10f-b95c-44c3-a259-6854b1cfde09'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:37:33.209438 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-8943c10f-b95c-44c3-a259-6854b1cfde09 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86651f48c1f441f1b0e115e57b840a28', 'request_id': 'req-8943c10f-b95c-44c3-a259-6854b1cfde09'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:37:33.220987 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-8943c10f-b95c-44c3-a259-6854b1cfde09 admin admin] The token cache is being invalidate because project 86651f48c1f441f1b0e115e57b840a28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:37:33.221871 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-8943c10f-b95c-44c3-a259-6854b1cfde09 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:37:33.222504 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-8943c10f-b95c-44c3-a259-6854b1cfde09 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:37:33.224016 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1156/2323] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Aug 6 01:37:33 2022] DELETE /identity/v3/projects/86651f48c1f441f1b0e115e57b840a28 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:37:33.228871 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46a7039-27e7-4aa7-8b40-fcb707932226 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:37:33.229367 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46a7039-27e7-4aa7-8b40-fcb707932226 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:37:33.229476 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46a7039-27e7-4aa7-8b40-fcb707932226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:37:33.249890 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-e46a7039-27e7-4aa7-8b40-fcb707932226 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:37:33.251100 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-e46a7039-27e7-4aa7-8b40-fcb707932226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:37:33.315880 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1168/2324] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:37:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:37:33.324011 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c20345-c65a-435d-b194-4f3ca1829c0e None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:37:33.324528 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c20345-c65a-435d-b194-4f3ca1829c0e None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:37:33.324787 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c20345-c65a-435d-b194-4f3ca1829c0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:37:33.344584 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-99c20345-c65a-435d-b194-4f3ca1829c0e None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:37:33.345824 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-99c20345-c65a-435d-b194-4f3ca1829c0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:37:33.354400 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1157/2325] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:37:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:37:33.358656 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46a7039-27e7-4aa7-8b40-fcb707932226 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:37:33.369659 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46a7039-27e7-4aa7-8b40-fcb707932226 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:37:33.374261 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2150a3a0-d8f3-4d3a-b660-9eb7634b639c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:37:33.374872 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2150a3a0-d8f3-4d3a-b660-9eb7634b639c admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:37:33.375157 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2150a3a0-d8f3-4d3a-b660-9eb7634b639c admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:37:33.375440 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2150a3a0-d8f3-4d3a-b660-9eb7634b639c admin admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:37:33.376137 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2150a3a0-d8f3-4d3a-b660-9eb7634b639c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:37:33.376443 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2150a3a0-d8f3-4d3a-b660-9eb7634b639c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:37:33.378505 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2150a3a0-d8f3-4d3a-b660-9eb7634b639c admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:37:33.389906 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1169/2326] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Aug 6 01:37:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:37:33.394836 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c20345-c65a-435d-b194-4f3ca1829c0e None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:37:33.404313 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c20345-c65a-435d-b194-4f3ca1829c0e None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:37:33.409954 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be0b4f2-e5ba-46f5-abe6-fbe5dc11d4d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:37:33.410582 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be0b4f2-e5ba-46f5-abe6-fbe5dc11d4d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:37:33.410868 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be0b4f2-e5ba-46f5-abe6-fbe5dc11d4d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:37:33.411149 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be0b4f2-e5ba-46f5-abe6-fbe5dc11d4d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:37:33.411827 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be0b4f2-e5ba-46f5-abe6-fbe5dc11d4d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:37:33.413776 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be0b4f2-e5ba-46f5-abe6-fbe5dc11d4d7 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:37:33.440775 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1158/2327] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Aug 6 01:37:33 2022] POST /identity/v3/projects => generated 397 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:37:33.445030 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2150a3a0-d8f3-4d3a-b660-9eb7634b639c admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:37:33.455069 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2150a3a0-d8f3-4d3a-b660-9eb7634b639c admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:37:33.515894 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0e0719-4c4b-4168-beef-94e254d3bec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:37:33.516530 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0e0719-4c4b-4168-beef-94e254d3bec0 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:37:33.516889 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0e0719-4c4b-4168-beef-94e254d3bec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:37:33.517267 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0e0719-4c4b-4168-beef-94e254d3bec0 admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:37:33.518452 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0e0719-4c4b-4168-beef-94e254d3bec0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:37:33.520425 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0e0719-4c4b-4168-beef-94e254d3bec0 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:37:33.546950 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1170/2328] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:37:33 2022] POST /identity/v3/users => generated 299 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:37:33.548684 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be0b4f2-e5ba-46f5-abe6-fbe5dc11d4d7 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:37:33.559359 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be0b4f2-e5ba-46f5-abe6-fbe5dc11d4d7 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:37:33.563988 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe81e251-b872-445b-8a0d-e94e533143f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:37:33.564603 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe81e251-b872-445b-8a0d-e94e533143f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:37:33.564920 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe81e251-b872-445b-8a0d-e94e533143f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:37:33.565252 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe81e251-b872-445b-8a0d-e94e533143f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:37:33.565935 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe81e251-b872-445b-8a0d-e94e533143f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:37:33.567516 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe81e251-b872-445b-8a0d-e94e533143f3 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:37:33.580258 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1159/2329] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:37:33 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:37:33.582358 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0e0719-4c4b-4168-beef-94e254d3bec0 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:37:33.592563 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0e0719-4c4b-4168-beef-94e254d3bec0 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:37:33.598225 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596e0c0a-b2b1-45fb-9d69-03391375f4f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:37:33.598841 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596e0c0a-b2b1-45fb-9d69-03391375f4f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:37:33.599125 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596e0c0a-b2b1-45fb-9d69-03391375f4f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:37:33.599466 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596e0c0a-b2b1-45fb-9d69-03391375f4f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:37:33.600177 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596e0c0a-b2b1-45fb-9d69-03391375f4f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:37:33.601804 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596e0c0a-b2b1-45fb-9d69-03391375f4f4 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:37:33.612806 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1171/2330] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:37:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:37:33.618249 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe81e251-b872-445b-8a0d-e94e533143f3 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:37:33.628502 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe81e251-b872-445b-8a0d-e94e533143f3 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:37:33.633235 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a2ddbe-b8bb-4d37-a99a-e916595d8b82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:37:33.633900 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a2ddbe-b8bb-4d37-a99a-e916595d8b82 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:37:33.634184 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a2ddbe-b8bb-4d37-a99a-e916595d8b82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:37:33.634462 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a2ddbe-b8bb-4d37-a99a-e916595d8b82 admin admin] PATH_INFO: `/v3/projects/da3db0ea62594d2a88789d79dbe0d241/users/eeaab19cce014db097527e7d0e24f953/roles/61b3b93d37a047e4bb8369b46fe20db9` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:37:33.649621 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a2ddbe-b8bb-4d37-a99a-e916595d8b82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da3db0ea62594d2a88789d79dbe0d241, user_id=eeaab19cce014db097527e7d0e24f953, role_id=61b3b93d37a047e4bb8369b46fe20db9)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:37:33.652288 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a2ddbe-b8bb-4d37-a99a-e916595d8b82 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:37:33.665889 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1160/2331] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:37:33 2022] PUT /identity/v3/projects/da3db0ea62594d2a88789d79dbe0d241/users/eeaab19cce014db097527e7d0e24f953/roles/61b3b93d37a047e4bb8369b46fe20db9 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:37:33.676084 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517252e1-5717-43b3-82a3-ca34c0cd3285 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:37:33.676378 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517252e1-5717-43b3-82a3-ca34c0cd3285 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:37:33.676661 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517252e1-5717-43b3-82a3-ca34c0cd3285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:37:33.703455 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-517252e1-5717-43b3-82a3-ca34c0cd3285 None None] MFA Rules not processed for user `eeaab19cce014db097527e7d0e24f953`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:37:33.704666 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-517252e1-5717-43b3-82a3-ca34c0cd3285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:37:33.746591 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:37:33.771622 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1172/2332] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:37:33 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:37:33.783009 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a2ddbe-b8bb-4d37-a99a-e916595d8b82 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:37:33.785532 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-a1a2ddbe-b8bb-4d37-a99a-e916595d8b82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:37:33.841681 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a2ddbe-b8bb-4d37-a99a-e916595d8b82 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:37:33.845294 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85263ba2-3178-417e-9377-5fc76920efb5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1be1e6173e374a60b9e1fbf51a1ab571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:37:33.845904 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85263ba2-3178-417e-9377-5fc76920efb5 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:37:33.846345 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85263ba2-3178-417e-9377-5fc76920efb5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:37:33.846628 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85263ba2-3178-417e-9377-5fc76920efb5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:37:33.854694 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85263ba2-3178-417e-9377-5fc76920efb5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:37:33.856104 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85263ba2-3178-417e-9377-5fc76920efb5 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:37:33.864664 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1161/2333] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:37:33 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:37:43.768036 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=8 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:37:43.860722 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=71,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:37:44.088267 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517252e1-5717-43b3-82a3-ca34c0cd3285 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:37:44.089683 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-517252e1-5717-43b3-82a3-ca34c0cd3285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:37:44.146203 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517252e1-5717-43b3-82a3-ca34c0cd3285 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:37:44.149784 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87c37d7-0f49-421f-b4e3-aa0a9f1e8539 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:37:44.150397 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87c37d7-0f49-421f-b4e3-aa0a9f1e8539 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:37:44.150679 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87c37d7-0f49-421f-b4e3-aa0a9f1e8539 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:37:44.150952 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87c37d7-0f49-421f-b4e3-aa0a9f1e8539 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:37:44.152684 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-e87c37d7-0f49-421f-b4e3-aa0a9f1e8539 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:37:44.222883 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87c37d7-0f49-421f-b4e3-aa0a9f1e8539 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:37:44.224314 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87c37d7-0f49-421f-b4e3-aa0a9f1e8539 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:37:44.235078 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1173/2334] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:37:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:37:44.560072 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85263ba2-3178-417e-9377-5fc76920efb5 service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:37:44.570126 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85263ba2-3178-417e-9377-5fc76920efb5 service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:37:44.573759 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6266eec4-3535-4ac6-b5ab-5571422b8c79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:37:44.574366 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6266eec4-3535-4ac6-b5ab-5571422b8c79 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:37:44.574649 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6266eec4-3535-4ac6-b5ab-5571422b8c79 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:37:44.574930 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6266eec4-3535-4ac6-b5ab-5571422b8c79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:37:44.576664 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-6266eec4-3535-4ac6-b5ab-5571422b8c79 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:37:44.591890 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6266eec4-3535-4ac6-b5ab-5571422b8c79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:37:44.593277 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6266eec4-3535-4ac6-b5ab-5571422b8c79 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:37:44.604307 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1162/2335] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:37:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:37:49.170976 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87c37d7-0f49-421f-b4e3-aa0a9f1e8539 service neutron] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:37:49.233442 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87c37d7-0f49-421f-b4e3-aa0a9f1e8539 service neutron] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:37:49.238754 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e8fd83-a116-4e71-bf16-0fbb50a20260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:37:49.239393 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e8fd83-a116-4e71-bf16-0fbb50a20260 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:37:49.239678 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e8fd83-a116-4e71-bf16-0fbb50a20260 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:37:49.239953 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e8fd83-a116-4e71-bf16-0fbb50a20260 admin admin] PATH_INFO: `/v3/users/eeaab19cce014db097527e7d0e24f953` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:37:49.241154 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e8fd83-a116-4e71-bf16-0fbb50a20260 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eeaab19cce014db097527e7d0e24f953)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:37:49.243376 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e8fd83-a116-4e71-bf16-0fbb50a20260 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:37:49.287332 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:37:49.290546 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-46e8fd83-a116-4e71-bf16-0fbb50a20260 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eeaab19cce014db097527e7d0e24f953'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:37:49.299432 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-46e8fd83-a116-4e71-bf16-0fbb50a20260 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eeaab19cce014db097527e7d0e24f953'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:37:49.301015 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-46e8fd83-a116-4e71-bf16-0fbb50a20260 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eeaab19cce014db097527e7d0e24f953'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:37:49.301947 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-46e8fd83-a116-4e71-bf16-0fbb50a20260 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eeaab19cce014db097527e7d0e24f953'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:37:49.314557 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-46e8fd83-a116-4e71-bf16-0fbb50a20260 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eeaab19cce014db097527e7d0e24f953'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:37:49.328074 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1174/2336] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Aug 6 01:37:49 2022] DELETE /identity/v3/users/eeaab19cce014db097527e7d0e24f953 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:37:49.332668 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb874a54-eb59-4d67-840b-0665b69b5968 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:37:49.333162 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb874a54-eb59-4d67-840b-0665b69b5968 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:37:49.333244 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb874a54-eb59-4d67-840b-0665b69b5968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:37:49.352837 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-fb874a54-eb59-4d67-840b-0665b69b5968 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:37:49.354089 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-fb874a54-eb59-4d67-840b-0665b69b5968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:37:49.424975 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1163/2337] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:37:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:37:49.435111 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e8fd83-a116-4e71-bf16-0fbb50a20260 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:37:49.436641 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-46e8fd83-a116-4e71-bf16-0fbb50a20260 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:37:49.497061 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e8fd83-a116-4e71-bf16-0fbb50a20260 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:37:49.500863 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa958c5-f817-4fa6-9321-c954cb026c5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:37:49.501545 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa958c5-f817-4fa6-9321-c954cb026c5f service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:37:49.501826 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa958c5-f817-4fa6-9321-c954cb026c5f service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:37:49.502100 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa958c5-f817-4fa6-9321-c954cb026c5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:37:49.512447 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa958c5-f817-4fa6-9321-c954cb026c5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:37:49.513906 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa958c5-f817-4fa6-9321-c954cb026c5f service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:37:49.524290 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1175/2338] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:37:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:37:49.852835 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb874a54-eb59-4d67-840b-0665b69b5968 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:37:49.854396 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-fb874a54-eb59-4d67-840b-0665b69b5968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:37:49.868901 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb874a54-eb59-4d67-840b-0665b69b5968 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:37:49.873649 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adf1300-cb21-4d66-8420-e499601432ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:37:49.874294 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adf1300-cb21-4d66-8420-e499601432ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:37:49.874579 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adf1300-cb21-4d66-8420-e499601432ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:37:49.874870 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adf1300-cb21-4d66-8420-e499601432ac admin admin] PATH_INFO: `/v3/projects/da3db0ea62594d2a88789d79dbe0d241` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:37:49.876957 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adf1300-cb21-4d66-8420-e499601432ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=da3db0ea62594d2a88789d79dbe0d241)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:37:49.879352 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adf1300-cb21-4d66-8420-e499601432ac admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:37:49.925635 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-4adf1300-cb21-4d66-8420-e499601432ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da3db0ea62594d2a88789d79dbe0d241', 'request_id': 'req-4adf1300-cb21-4d66-8420-e499601432ac'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:37:49.931659 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-4adf1300-cb21-4d66-8420-e499601432ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da3db0ea62594d2a88789d79dbe0d241', 'request_id': 'req-4adf1300-cb21-4d66-8420-e499601432ac'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:37:49.937361 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-4adf1300-cb21-4d66-8420-e499601432ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da3db0ea62594d2a88789d79dbe0d241', 'request_id': 'req-4adf1300-cb21-4d66-8420-e499601432ac'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:37:49.950593 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-4adf1300-cb21-4d66-8420-e499601432ac admin admin] The token cache is being invalidate because project da3db0ea62594d2a88789d79dbe0d241 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:37:49.950909 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-4adf1300-cb21-4d66-8420-e499601432ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:37:49.951518 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-4adf1300-cb21-4d66-8420-e499601432ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:37:49.953002 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1164/2339] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Aug 6 01:37:49 2022] DELETE /identity/v3/projects/da3db0ea62594d2a88789d79dbe0d241 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:37:59.519842 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:DELETE=4,keystone:SELECT=39,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:37:59.947446 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=60,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:38:02.995355 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa958c5-f817-4fa6-9321-c954cb026c5f service neutron] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:38:02.996738 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-8fa958c5-f817-4fa6-9321-c954cb026c5f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:38:03.042894 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adf1300-cb21-4d66-8420-e499601432ac admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:38:03.045532 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-4adf1300-cb21-4d66-8420-e499601432ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:38:03.064464 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa958c5-f817-4fa6-9321-c954cb026c5f service neutron] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:38:03.129970 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3911f4-658b-4c14-939b-1f65367b9451 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abb76a5c8ef74bf2b18c56074daaad58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:38:03.130600 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3911f4-658b-4c14-939b-1f65367b9451 service glance] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:38:03.130883 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3911f4-658b-4c14-939b-1f65367b9451 service glance] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:38:03.131308 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3911f4-658b-4c14-939b-1f65367b9451 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:38:03.133105 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-ad3911f4-658b-4c14-939b-1f65367b9451 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:38:03.154061 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adf1300-cb21-4d66-8420-e499601432ac admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:38:03.162288 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3e1dd9-55c1-4fcd-b242-f3af24ef9c68 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87adc1d5bc240e99f95f6595b350be4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:38:03.163483 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3e1dd9-55c1-4fcd-b242-f3af24ef9c68 service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:38:03.163609 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3e1dd9-55c1-4fcd-b242-f3af24ef9c68 service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:38:03.163924 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3e1dd9-55c1-4fcd-b242-f3af24ef9c68 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:38:03.166293 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-7a3e1dd9-55c1-4fcd-b242-f3af24ef9c68 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:38:03.204324 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3911f4-658b-4c14-939b-1f65367b9451 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:38:03.205907 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3911f4-658b-4c14-939b-1f65367b9451 service glance] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:38:03.216129 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1176/2340] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:38:02 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:38:03.229924 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3e1dd9-55c1-4fcd-b242-f3af24ef9c68 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:38:03.231453 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3e1dd9-55c1-4fcd-b242-f3af24ef9c68 service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:38:03.242774 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1165/2341] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:38:03 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 01:38:03.649534 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d2fc1b-93c1-49e1-8fcb-282535a66372 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:38:03.649732 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d2fc1b-93c1-49e1-8fcb-282535a66372 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:38:03.649993 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d2fc1b-93c1-49e1-8fcb-282535a66372 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:38:03.670848 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-27d2fc1b-93c1-49e1-8fcb-282535a66372 None None] MFA Rules not processed for user `8e86f38590824894a3b960e6774ff8a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:38:03.680094 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-27d2fc1b-93c1-49e1-8fcb-282535a66372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:38:03.752410 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1177/2342] 10.222.0.26 () {58 vars in 1009 bytes} [Sat Aug 6 01:38:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:38:03.765189 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3e1dd9-55c1-4fcd-b242-f3af24ef9c68 service nova] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:38:03.766612 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-7a3e1dd9-55c1-4fcd-b242-f3af24ef9c68 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:38:03.840808 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3e1dd9-55c1-4fcd-b242-f3af24ef9c68 service nova] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:38:03.845865 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979242eb-1346-4160-b127-75c5f7f45f48 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b23f4478f49471fba6f43e6e4be99da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:38:03.846480 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979242eb-1346-4160-b127-75c5f7f45f48 service swift] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:38:03.846764 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979242eb-1346-4160-b127-75c5f7f45f48 service swift] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:38:03.847045 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979242eb-1346-4160-b127-75c5f7f45f48 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:38:03.855810 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979242eb-1346-4160-b127-75c5f7f45f48 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:38:03.857408 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979242eb-1346-4160-b127-75c5f7f45f48 service swift] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:38:03.865516 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1166/2343] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Aug 6 01:38:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 06 01:38:13.744118 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=93 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:38:13.862079 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=88 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:38:14.425635 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d2fc1b-93c1-49e1-8fcb-282535a66372 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:38:14.427865 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-27d2fc1b-93c1-49e1-8fcb-282535a66372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:38:14.496746 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d2fc1b-93c1-49e1-8fcb-282535a66372 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:38:14.500715 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80a6db2-5e29-4e46-9f8d-154e279b6ebf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:38:14.501427 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80a6db2-5e29-4e46-9f8d-154e279b6ebf service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:38:14.501675 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80a6db2-5e29-4e46-9f8d-154e279b6ebf service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:38:14.501959 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80a6db2-5e29-4e46-9f8d-154e279b6ebf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:38:14.504406 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-c80a6db2-5e29-4e46-9f8d-154e279b6ebf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:38:14.514710 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80a6db2-5e29-4e46-9f8d-154e279b6ebf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:38:14.516917 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80a6db2-5e29-4e46-9f8d-154e279b6ebf service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:38:14.527522 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1178/2344] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:38:14 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:38:21.840698 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979242eb-1346-4160-b127-75c5f7f45f48 service swift] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:38:21.850094 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979242eb-1346-4160-b127-75c5f7f45f48 service swift] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:38:21.853765 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f33ff2-ef7a-4948-b884-9899a27d846f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:38:21.854385 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f33ff2-ef7a-4948-b884-9899a27d846f service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:38:21.854666 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f33ff2-ef7a-4948-b884-9899a27d846f service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:38:21.854943 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f33ff2-ef7a-4948-b884-9899a27d846f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:38:21.856717 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-61f33ff2-ef7a-4948-b884-9899a27d846f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:38:21.866985 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f33ff2-ef7a-4948-b884-9899a27d846f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:38:21.868704 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f33ff2-ef7a-4948-b884-9899a27d846f service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:38:21.877136 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1167/2345] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:38:21 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:38:22.001879 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95bdf08-77b4-42b2-ba32-cb7f48ced3ae None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:38:22.002865 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95bdf08-77b4-42b2-ba32-cb7f48ced3ae None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:38:22.003247 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95bdf08-77b4-42b2-ba32-cb7f48ced3ae None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:38:22.005557 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1179/2346] 10.222.0.26 () {54 vars in 921 bytes} [Sat Aug 6 01:38:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:38:22.012483 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6768c0c1-7e60-48fa-a1ac-8b5e29fbfd84 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:38:22.012808 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6768c0c1-7e60-48fa-a1ac-8b5e29fbfd84 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:38:22.013142 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6768c0c1-7e60-48fa-a1ac-8b5e29fbfd84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:38:22.033865 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-6768c0c1-7e60-48fa-a1ac-8b5e29fbfd84 None None] MFA Rules not processed for user `b87adc1d5bc240e99f95f6595b350be4`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:38:22.035383 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-6768c0c1-7e60-48fa-a1ac-8b5e29fbfd84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:38:22.045249 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1168/2347] 10.222.0.26 () {58 vars in 1018 bytes} [Sat Aug 6 01:38:22 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:38:22.057588 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95bdf08-77b4-42b2-ba32-cb7f48ced3ae None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:38:22.060231 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-b95bdf08-77b4-42b2-ba32-cb7f48ced3ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:38:22.075535 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95bdf08-77b4-42b2-ba32-cb7f48ced3ae None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:38:22.079530 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb945302-230c-4c73-b9ef-21313c2d0b30 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abb76a5c8ef74bf2b18c56074daaad58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:38:22.080219 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb945302-230c-4c73-b9ef-21313c2d0b30 service glance] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:38:22.080616 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb945302-230c-4c73-b9ef-21313c2d0b30 service glance] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:38:22.080977 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb945302-230c-4c73-b9ef-21313c2d0b30 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:38:22.092396 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb945302-230c-4c73-b9ef-21313c2d0b30 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:38:22.093968 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb945302-230c-4c73-b9ef-21313c2d0b30 service glance] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:38:22.105502 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1180/2348] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:38:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:38:22.192238 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e61dd8-085e-4fa2-b1b4-50ff205d1d09 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:38:22.192590 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e61dd8-085e-4fa2-b1b4-50ff205d1d09 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:38:22.192827 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e61dd8-085e-4fa2-b1b4-50ff205d1d09 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:38:22.194450 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1169/2349] 10.222.0.26 () {54 vars in 921 bytes} [Sat Aug 6 01:38:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:38:22.199506 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2973790e-61d3-448b-852c-9b24cb52e597 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:38:22.199799 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2973790e-61d3-448b-852c-9b24cb52e597 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:38:22.200095 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2973790e-61d3-448b-852c-9b24cb52e597 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:38:22.221690 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-2973790e-61d3-448b-852c-9b24cb52e597 None None] MFA Rules not processed for user `4ac0e31c84b4445fa167bbe4954066c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:38:22.223601 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-2973790e-61d3-448b-852c-9b24cb52e597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:38:22.230346 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1181/2350] 10.222.0.26 () {58 vars in 1018 bytes} [Sat Aug 6 01:38:22 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:38:22.242913 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e61dd8-085e-4fa2-b1b4-50ff205d1d09 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:38:22.244612 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-c6e61dd8-085e-4fa2-b1b4-50ff205d1d09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:38:22.258539 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e61dd8-085e-4fa2-b1b4-50ff205d1d09 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:38:22.262298 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54a1304-5fae-4a16-aed2-daa57d953ecc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:38:22.262918 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54a1304-5fae-4a16-aed2-daa57d953ecc service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:38:22.263205 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54a1304-5fae-4a16-aed2-daa57d953ecc service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:38:22.263487 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54a1304-5fae-4a16-aed2-daa57d953ecc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:38:22.271034 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54a1304-5fae-4a16-aed2-daa57d953ecc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:38:22.272476 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54a1304-5fae-4a16-aed2-daa57d953ecc service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:38:22.280891 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1170/2351] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:38:22 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:38:22.372081 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cb2513-1fa2-4ad1-b63d-0e76d87df130 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:38:22.372372 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cb2513-1fa2-4ad1-b63d-0e76d87df130 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:38:22.372653 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cb2513-1fa2-4ad1-b63d-0e76d87df130 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:38:22.374078 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1182/2352] 10.222.0.26 () {54 vars in 921 bytes} [Sat Aug 6 01:38:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:38:22.387609 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37f9a3b-9712-4a29-a837-b6493568fbb0 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:38:22.388051 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37f9a3b-9712-4a29-a837-b6493568fbb0 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:38:22.388343 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37f9a3b-9712-4a29-a837-b6493568fbb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:38:22.411381 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-a37f9a3b-9712-4a29-a837-b6493568fbb0 None None] MFA Rules not processed for user `b87adc1d5bc240e99f95f6595b350be4`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:38:22.412698 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-a37f9a3b-9712-4a29-a837-b6493568fbb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:38:22.423530 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1171/2353] 10.222.0.26 () {58 vars in 1018 bytes} [Sat Aug 6 01:38:22 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:38:22.435289 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cb2513-1fa2-4ad1-b63d-0e76d87df130 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:38:22.446938 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cb2513-1fa2-4ad1-b63d-0e76d87df130 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:38:22.450521 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b9528d-ea95-4f49-b03b-0252b31d273f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:38:22.451128 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b9528d-ea95-4f49-b03b-0252b31d273f service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:38:22.451407 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b9528d-ea95-4f49-b03b-0252b31d273f service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:38:22.451678 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b9528d-ea95-4f49-b03b-0252b31d273f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:38:22.461871 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b9528d-ea95-4f49-b03b-0252b31d273f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:38:22.463286 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b9528d-ea95-4f49-b03b-0252b31d273f service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:38:22.474927 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1183/2354] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:38:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:38:22.538043 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37f9a3b-9712-4a29-a837-b6493568fbb0 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:38:22.539505 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-a37f9a3b-9712-4a29-a837-b6493568fbb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:38:22.607365 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37f9a3b-9712-4a29-a837-b6493568fbb0 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:38:22.612317 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41482fb8-5755-4f11-af0f-4f1b45eb7a28 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa3b93ddd9464a2cba724d67b56b477a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:38:22.612951 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41482fb8-5755-4f11-af0f-4f1b45eb7a28 service placement] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:38:22.613289 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41482fb8-5755-4f11-af0f-4f1b45eb7a28 service placement] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:38:22.613563 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41482fb8-5755-4f11-af0f-4f1b45eb7a28 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:38:22.615318 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-41482fb8-5755-4f11-af0f-4f1b45eb7a28 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:38:22.626894 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41482fb8-5755-4f11-af0f-4f1b45eb7a28 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:38:22.628673 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41482fb8-5755-4f11-af0f-4f1b45eb7a28 service placement] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:38:22.638334 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1172/2355] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:38:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 01:38:25.434584 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce9e87c-abd2-4656-821f-37d845fec24a None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:38:25.435035 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce9e87c-abd2-4656-821f-37d845fec24a None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:38:25.435139 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce9e87c-abd2-4656-821f-37d845fec24a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:38:25.453875 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-0ce9e87c-abd2-4656-821f-37d845fec24a None None] MFA Rules not processed for user `8e86f38590824894a3b960e6774ff8a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:38:25.455089 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-0ce9e87c-abd2-4656-821f-37d845fec24a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:38:25.462096 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1184/2356] 10.222.0.26 () {58 vars in 1009 bytes} [Sat Aug 6 01:38:25 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:38:25.474500 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41482fb8-5755-4f11-af0f-4f1b45eb7a28 service placement] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:38:25.485527 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41482fb8-5755-4f11-af0f-4f1b45eb7a28 service placement] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:38:25.491301 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62460e2-96a0-4b16-a38d-0247df1cd20e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b23f4478f49471fba6f43e6e4be99da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:38:25.491912 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62460e2-96a0-4b16-a38d-0247df1cd20e service swift] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:38:25.492191 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62460e2-96a0-4b16-a38d-0247df1cd20e service swift] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:38:25.492719 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62460e2-96a0-4b16-a38d-0247df1cd20e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:38:25.500772 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62460e2-96a0-4b16-a38d-0247df1cd20e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:38:25.502369 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62460e2-96a0-4b16-a38d-0247df1cd20e service swift] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:38:25.510394 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1173/2357] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Aug 6 01:38:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 06 01:38:32.289211 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce9e87c-abd2-4656-821f-37d845fec24a None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:38:32.289832 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-0ce9e87c-abd2-4656-821f-37d845fec24a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:38:32.378870 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce9e87c-abd2-4656-821f-37d845fec24a None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:38:32.396009 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69c75e6-8ce9-45b5-b732-07cdb0c01898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:38:32.396009 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69c75e6-8ce9-45b5-b732-07cdb0c01898 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:38:32.396009 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69c75e6-8ce9-45b5-b732-07cdb0c01898 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:38:32.396009 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69c75e6-8ce9-45b5-b732-07cdb0c01898 admin admin] PATH_INFO: `/v3/users/c938a575a2704078a82fe6df941884dc` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:38:32.409959 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69c75e6-8ce9-45b5-b732-07cdb0c01898 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c938a575a2704078a82fe6df941884dc)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:38:32.412786 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69c75e6-8ce9-45b5-b732-07cdb0c01898 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:38:32.467776 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-c69c75e6-8ce9-45b5-b732-07cdb0c01898 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c938a575a2704078a82fe6df941884dc'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:38:32.477578 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-c69c75e6-8ce9-45b5-b732-07cdb0c01898 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c938a575a2704078a82fe6df941884dc'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:38:32.478588 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-c69c75e6-8ce9-45b5-b732-07cdb0c01898 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c938a575a2704078a82fe6df941884dc'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:38:32.479548 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-c69c75e6-8ce9-45b5-b732-07cdb0c01898 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c938a575a2704078a82fe6df941884dc'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:38:32.488462 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:38:32.497647 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-c69c75e6-8ce9-45b5-b732-07cdb0c01898 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c938a575a2704078a82fe6df941884dc'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:38:32.517850 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1185/2358] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Aug 6 01:38:32 2022] DELETE /identity/v3/users/c938a575a2704078a82fe6df941884dc => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:38:32.726186 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62460e2-96a0-4b16-a38d-0247df1cd20e service swift] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:38:32.749513 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62460e2-96a0-4b16-a38d-0247df1cd20e service swift] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:38:32.835526 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbea578-7771-4996-983f-c3221d8141af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:38:32.836274 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbea578-7771-4996-983f-c3221d8141af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:38:32.836991 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbea578-7771-4996-983f-c3221d8141af admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:38:32.836991 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbea578-7771-4996-983f-c3221d8141af admin admin] PATH_INFO: `/v3/projects/e50641dc16fa4187b5f0d4f936d34d53` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:38:32.838375 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbea578-7771-4996-983f-c3221d8141af admin admin] RBAC: Authorizing `identity:delete_project(project_id=e50641dc16fa4187b5f0d4f936d34d53)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:38:32.840486 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbea578-7771-4996-983f-c3221d8141af admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:38:32.922019 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:38:32.930760 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-9fbea578-7771-4996-983f-c3221d8141af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e50641dc16fa4187b5f0d4f936d34d53', 'request_id': 'req-9fbea578-7771-4996-983f-c3221d8141af'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:38:32.939502 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-9fbea578-7771-4996-983f-c3221d8141af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e50641dc16fa4187b5f0d4f936d34d53', 'request_id': 'req-9fbea578-7771-4996-983f-c3221d8141af'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:38:32.946286 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-9fbea578-7771-4996-983f-c3221d8141af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e50641dc16fa4187b5f0d4f936d34d53', 'request_id': 'req-9fbea578-7771-4996-983f-c3221d8141af'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:38:32.960497 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-9fbea578-7771-4996-983f-c3221d8141af admin admin] The token cache is being invalidate because project e50641dc16fa4187b5f0d4f936d34d53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:38:32.960790 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-9fbea578-7771-4996-983f-c3221d8141af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:38:32.961420 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-9fbea578-7771-4996-983f-c3221d8141af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:38:32.962957 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1174/2359] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Aug 6 01:38:32 2022] DELETE /identity/v3/projects/e50641dc16fa4187b5f0d4f936d34d53 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:38:32.968604 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2595519-09c8-447c-9cee-e23faa6f40f0 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:38:32.968824 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2595519-09c8-447c-9cee-e23faa6f40f0 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:38:32.969192 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2595519-09c8-447c-9cee-e23faa6f40f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:38:32.996009 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-a2595519-09c8-447c-9cee-e23faa6f40f0 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:38:32.997834 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-a2595519-09c8-447c-9cee-e23faa6f40f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:38:33.066729 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1186/2360] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:38:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:38:33.074786 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a9d87e-190b-4312-b0ba-f5f9dbe439ca None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:38:33.075072 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a9d87e-190b-4312-b0ba-f5f9dbe439ca None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:38:33.075348 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a9d87e-190b-4312-b0ba-f5f9dbe439ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:38:33.094882 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-86a9d87e-190b-4312-b0ba-f5f9dbe439ca None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:38:33.096086 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-86a9d87e-190b-4312-b0ba-f5f9dbe439ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:38:33.104001 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1175/2361] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:38:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:38:33.108078 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2595519-09c8-447c-9cee-e23faa6f40f0 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:38:33.118838 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2595519-09c8-447c-9cee-e23faa6f40f0 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:38:33.124198 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a999095-d142-4888-8a35-0dda19c0def3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:38:33.125240 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a999095-d142-4888-8a35-0dda19c0def3 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:38:33.125857 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a999095-d142-4888-8a35-0dda19c0def3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:38:33.126236 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a999095-d142-4888-8a35-0dda19c0def3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:38:33.127412 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a999095-d142-4888-8a35-0dda19c0def3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:38:33.127724 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a999095-d142-4888-8a35-0dda19c0def3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:38:33.133087 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a999095-d142-4888-8a35-0dda19c0def3 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:38:33.149358 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1187/2362] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Aug 6 01:38:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:38:33.150263 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a9d87e-190b-4312-b0ba-f5f9dbe439ca None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:38:33.163842 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a9d87e-190b-4312-b0ba-f5f9dbe439ca None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:38:33.172023 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445d2fed-d54e-4338-a524-89f9f6c6bf89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:38:33.173613 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445d2fed-d54e-4338-a524-89f9f6c6bf89 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:38:33.175414 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445d2fed-d54e-4338-a524-89f9f6c6bf89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:38:33.175800 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445d2fed-d54e-4338-a524-89f9f6c6bf89 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:38:33.177951 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445d2fed-d54e-4338-a524-89f9f6c6bf89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:38:33.180205 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445d2fed-d54e-4338-a524-89f9f6c6bf89 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:38:33.230197 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1176/2363] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Aug 6 01:38:33 2022] POST /identity/v3/projects => generated 385 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:38:33.234251 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a999095-d142-4888-8a35-0dda19c0def3 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:38:33.245457 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a999095-d142-4888-8a35-0dda19c0def3 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:38:33.331332 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ce9836-e579-414e-b615-c0aed0480f3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:38:33.331943 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ce9836-e579-414e-b615-c0aed0480f3f admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:38:33.332231 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ce9836-e579-414e-b615-c0aed0480f3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:38:33.332690 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ce9836-e579-414e-b615-c0aed0480f3f admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:38:33.333267 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ce9836-e579-414e-b615-c0aed0480f3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:38:33.337466 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ce9836-e579-414e-b615-c0aed0480f3f admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:38:33.359662 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1188/2364] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:38:33 2022] POST /identity/v3/users => generated 293 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:38:33.364384 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445d2fed-d54e-4338-a524-89f9f6c6bf89 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:38:33.375007 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445d2fed-d54e-4338-a524-89f9f6c6bf89 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:38:33.380375 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9eceb1-fea8-46fe-bded-185c97b1c181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:38:33.380987 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9eceb1-fea8-46fe-bded-185c97b1c181 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:38:33.381304 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9eceb1-fea8-46fe-bded-185c97b1c181 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:38:33.381642 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9eceb1-fea8-46fe-bded-185c97b1c181 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:38:33.382314 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9eceb1-fea8-46fe-bded-185c97b1c181 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:38:33.383911 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9eceb1-fea8-46fe-bded-185c97b1c181 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:38:33.393933 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1177/2365] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:38:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:38:33.398170 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ce9836-e579-414e-b615-c0aed0480f3f admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:38:33.408167 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ce9836-e579-414e-b615-c0aed0480f3f admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:38:33.413437 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956aefbc-709e-44c2-9647-1db30f03dfbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:38:33.414053 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956aefbc-709e-44c2-9647-1db30f03dfbc admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:38:33.414337 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956aefbc-709e-44c2-9647-1db30f03dfbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:38:33.414614 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956aefbc-709e-44c2-9647-1db30f03dfbc admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:38:33.415313 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956aefbc-709e-44c2-9647-1db30f03dfbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:38:33.416914 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956aefbc-709e-44c2-9647-1db30f03dfbc admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:38:33.428887 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1189/2366] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:38:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 01:38:33.434423 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9eceb1-fea8-46fe-bded-185c97b1c181 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:38:33.446155 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9eceb1-fea8-46fe-bded-185c97b1c181 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:38:33.451811 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebef86e-5af5-4521-95f5-051e6192cf70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:38:33.452896 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebef86e-5af5-4521-95f5-051e6192cf70 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:38:33.453294 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebef86e-5af5-4521-95f5-051e6192cf70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:38:33.453647 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebef86e-5af5-4521-95f5-051e6192cf70 admin admin] PATH_INFO: `/v3/projects/4db9537bd65748f6b43061f3530f6891/users/5cb73f6047dd42aaba23072f51972e9f/roles/61b3b93d37a047e4bb8369b46fe20db9` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:38:33.471364 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebef86e-5af5-4521-95f5-051e6192cf70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4db9537bd65748f6b43061f3530f6891, user_id=5cb73f6047dd42aaba23072f51972e9f, role_id=61b3b93d37a047e4bb8369b46fe20db9)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:38:33.474739 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebef86e-5af5-4521-95f5-051e6192cf70 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:38:33.488189 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1178/2367] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:38:33 2022] PUT /identity/v3/projects/4db9537bd65748f6b43061f3530f6891/users/5cb73f6047dd42aaba23072f51972e9f/roles/61b3b93d37a047e4bb8369b46fe20db9 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:38:33.494658 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48967d2d-4de7-41c4-a4b7-5815ac292618 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:38:33.495020 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48967d2d-4de7-41c4-a4b7-5815ac292618 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:38:33.495371 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48967d2d-4de7-41c4-a4b7-5815ac292618 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:38:33.514717 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-48967d2d-4de7-41c4-a4b7-5815ac292618 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:38:33.515939 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-48967d2d-4de7-41c4-a4b7-5815ac292618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:38:33.561230 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:38:33.578585 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1190/2368] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:38:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:38:33.588213 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebef86e-5af5-4521-95f5-051e6192cf70 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:38:33.589945 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-aebef86e-5af5-4521-95f5-051e6192cf70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:38:33.654997 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebef86e-5af5-4521-95f5-051e6192cf70 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:38:33.658825 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6da2bec-fa20-4810-8d5c-78935fffb74c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:38:33.659452 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6da2bec-fa20-4810-8d5c-78935fffb74c service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:38:33.659803 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6da2bec-fa20-4810-8d5c-78935fffb74c service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:38:33.660078 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6da2bec-fa20-4810-8d5c-78935fffb74c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:38:33.670621 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6da2bec-fa20-4810-8d5c-78935fffb74c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:38:33.672065 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6da2bec-fa20-4810-8d5c-78935fffb74c service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:38:33.682963 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1179/2369] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:38:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:38:38.151937 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a9b33c-5a83-48d7-bf12-5f6bd4f7ff87 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:38:38.152532 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a9b33c-5a83-48d7-bf12-5f6bd4f7ff87 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:38:38.152532 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a9b33c-5a83-48d7-bf12-5f6bd4f7ff87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:38:38.183207 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-a8a9b33c-5a83-48d7-bf12-5f6bd4f7ff87 None None] MFA Rules not processed for user `5cb73f6047dd42aaba23072f51972e9f`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:38:38.184429 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-a8a9b33c-5a83-48d7-bf12-5f6bd4f7ff87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:38:38.270811 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1191/2370] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:38:38 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:38:38.275782 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6da2bec-fa20-4810-8d5c-78935fffb74c service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:38:38.286637 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6da2bec-fa20-4810-8d5c-78935fffb74c service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:38:38.291341 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb4bbe0-0211-42fe-a545-4a0a210338ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:38:38.291966 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb4bbe0-0211-42fe-a545-4a0a210338ca admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:38:38.292250 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb4bbe0-0211-42fe-a545-4a0a210338ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:38:38.292527 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb4bbe0-0211-42fe-a545-4a0a210338ca admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:38:38.293230 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb4bbe0-0211-42fe-a545-4a0a210338ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:38:38.295140 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb4bbe0-0211-42fe-a545-4a0a210338ca admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:38:38.325752 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1180/2371] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Aug 6 01:38:38 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:38:38.329739 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a9b33c-5a83-48d7-bf12-5f6bd4f7ff87 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:38:38.340879 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a9b33c-5a83-48d7-bf12-5f6bd4f7ff87 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:38:38.431908 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8495bc5-4148-4bdb-9048-209ce17477a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:38:38.432550 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8495bc5-4148-4bdb-9048-209ce17477a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:38:38.432830 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8495bc5-4148-4bdb-9048-209ce17477a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:38:38.433181 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8495bc5-4148-4bdb-9048-209ce17477a0 admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:38:38.433855 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8495bc5-4148-4bdb-9048-209ce17477a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:38:38.435814 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8495bc5-4148-4bdb-9048-209ce17477a0 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:38:38.465867 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1192/2372] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:38:38 2022] POST /identity/v3/users => generated 299 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:38:38.469745 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb4bbe0-0211-42fe-a545-4a0a210338ca admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:38:38.485375 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb4bbe0-0211-42fe-a545-4a0a210338ca admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:38:38.490551 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfdbcad-1985-4389-86d7-9c54a2fbaeec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:38:38.491184 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfdbcad-1985-4389-86d7-9c54a2fbaeec admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:38:38.491487 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfdbcad-1985-4389-86d7-9c54a2fbaeec admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:38:38.491756 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfdbcad-1985-4389-86d7-9c54a2fbaeec admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:38:38.492725 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfdbcad-1985-4389-86d7-9c54a2fbaeec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:38:38.494465 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfdbcad-1985-4389-86d7-9c54a2fbaeec admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:38:38.507221 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1181/2373] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:38:38 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 01:38:38.511449 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8495bc5-4148-4bdb-9048-209ce17477a0 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:38:38.523648 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8495bc5-4148-4bdb-9048-209ce17477a0 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:38:38.528784 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ecbe35-98da-4754-ad2d-3f65d2ed6400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:38:38.529532 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ecbe35-98da-4754-ad2d-3f65d2ed6400 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:38:38.529962 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ecbe35-98da-4754-ad2d-3f65d2ed6400 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:38:38.530245 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ecbe35-98da-4754-ad2d-3f65d2ed6400 admin admin] PATH_INFO: `/v3/projects/db7fea8cbb1b4c8b98849254635c1916/users/9c188b9cd883443ba28b1db11ccf555d/roles/4559b9a0837249f895a13c70096f8d38` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:38:38.547639 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ecbe35-98da-4754-ad2d-3f65d2ed6400 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db7fea8cbb1b4c8b98849254635c1916, user_id=9c188b9cd883443ba28b1db11ccf555d, role_id=4559b9a0837249f895a13c70096f8d38)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:38:38.550340 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ecbe35-98da-4754-ad2d-3f65d2ed6400 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:38:38.564392 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1193/2374] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:38:38 2022] PUT /identity/v3/projects/db7fea8cbb1b4c8b98849254635c1916/users/9c188b9cd883443ba28b1db11ccf555d/roles/4559b9a0837249f895a13c70096f8d38 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:38:38.569114 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfdbcad-1985-4389-86d7-9c54a2fbaeec admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:38:38.580236 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfdbcad-1985-4389-86d7-9c54a2fbaeec admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:38:38.611780 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:38:38.642742 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ee9897-ff6b-4cb4-a873-028f9f77d164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:38:38.643383 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ee9897-ff6b-4cb4-a873-028f9f77d164 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:38:38.643662 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ee9897-ff6b-4cb4-a873-028f9f77d164 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:38:38.643943 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ee9897-ff6b-4cb4-a873-028f9f77d164 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:38:38.644723 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ee9897-ff6b-4cb4-a873-028f9f77d164 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:38:38.646525 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ee9897-ff6b-4cb4-a873-028f9f77d164 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:38:38.657010 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1182/2375] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:38:38 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:38:38.661900 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ecbe35-98da-4754-ad2d-3f65d2ed6400 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:38:38.672436 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ecbe35-98da-4754-ad2d-3f65d2ed6400 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:38:38.678183 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa084dea-6285-4567-bd84-c69a012bb42b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:38:38.678910 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa084dea-6285-4567-bd84-c69a012bb42b admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:38:38.679189 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa084dea-6285-4567-bd84-c69a012bb42b admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:38:38.679481 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa084dea-6285-4567-bd84-c69a012bb42b admin admin] PATH_INFO: `/v3/projects/db7fea8cbb1b4c8b98849254635c1916/users/9c188b9cd883443ba28b1db11ccf555d/roles/4559b9a0837249f895a13c70096f8d38` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:38:38.681504 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa084dea-6285-4567-bd84-c69a012bb42b admin admin] RBAC: Authorizing `identity:create_grant(project_id=db7fea8cbb1b4c8b98849254635c1916, user_id=9c188b9cd883443ba28b1db11ccf555d, role_id=4559b9a0837249f895a13c70096f8d38)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:38:38.684168 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa084dea-6285-4567-bd84-c69a012bb42b admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:38:38.695297 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1194/2376] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:38:38 2022] PUT /identity/v3/projects/db7fea8cbb1b4c8b98849254635c1916/users/9c188b9cd883443ba28b1db11ccf555d/roles/4559b9a0837249f895a13c70096f8d38 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:38:43.292343 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc4a470-cf13-499c-b2db-c4b3ac5a1f46 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:38:43.292823 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc4a470-cf13-499c-b2db-c4b3ac5a1f46 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:38:43.292935 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc4a470-cf13-499c-b2db-c4b3ac5a1f46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:38:43.325645 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-9fc4a470-cf13-499c-b2db-c4b3ac5a1f46 None None] MFA Rules not processed for user `9c188b9cd883443ba28b1db11ccf555d`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:38:43.326918 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-9fc4a470-cf13-499c-b2db-c4b3ac5a1f46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:38:43.424936 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1183/2377] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:38:43 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:38:43.438021 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa084dea-6285-4567-bd84-c69a012bb42b admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:38:43.440048 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-fa084dea-6285-4567-bd84-c69a012bb42b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:38:43.479554 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:38:43.505223 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa084dea-6285-4567-bd84-c69a012bb42b admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:38:43.508996 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d60391-5284-4073-9490-4c03b4a1d90a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abb76a5c8ef74bf2b18c56074daaad58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:38:43.509693 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d60391-5284-4073-9490-4c03b4a1d90a service glance] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:38:43.509957 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d60391-5284-4073-9490-4c03b4a1d90a service glance] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:38:43.510230 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d60391-5284-4073-9490-4c03b4a1d90a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:38:43.569955 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d60391-5284-4073-9490-4c03b4a1d90a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:38:43.571469 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d60391-5284-4073-9490-4c03b4a1d90a service glance] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:38:43.581453 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1195/2378] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:38:43 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:38:43.719543 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beba339-f932-40cd-8968-a25ac3177fce None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:38:43.719840 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beba339-f932-40cd-8968-a25ac3177fce None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:38:43.720120 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beba339-f932-40cd-8968-a25ac3177fce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:38:43.743948 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-4beba339-f932-40cd-8968-a25ac3177fce None None] MFA Rules not processed for user `8e86f38590824894a3b960e6774ff8a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:38:43.745444 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-4beba339-f932-40cd-8968-a25ac3177fce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:38:43.849181 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1184/2379] 10.222.0.26 () {58 vars in 1009 bytes} [Sat Aug 6 01:38:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:38:43.886468 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d60391-5284-4073-9490-4c03b4a1d90a service glance] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:38:43.888240 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-c4d60391-5284-4073-9490-4c03b4a1d90a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:38:43.969828 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d60391-5284-4073-9490-4c03b4a1d90a service glance] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:38:43.975373 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec8fdb5-6826-4f56-ae77-607d65663887 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b23f4478f49471fba6f43e6e4be99da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:38:43.976085 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec8fdb5-6826-4f56-ae77-607d65663887 service swift] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:38:43.976441 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec8fdb5-6826-4f56-ae77-607d65663887 service swift] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:38:43.976805 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec8fdb5-6826-4f56-ae77-607d65663887 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:38:43.985823 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec8fdb5-6826-4f56-ae77-607d65663887 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:38:43.987441 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec8fdb5-6826-4f56-ae77-607d65663887 service swift] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:38:43.996980 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1196/2380] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Aug 6 01:38:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 06 01:38:49.492924 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4beba339-f932-40cd-8968-a25ac3177fce None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:38:49.495219 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-4beba339-f932-40cd-8968-a25ac3177fce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:38:49.607545 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:38:49.628337 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4beba339-f932-40cd-8968-a25ac3177fce None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:38:49.635426 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf04d35e-fa03-4b9d-a74d-f1185d85c463 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87adc1d5bc240e99f95f6595b350be4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:38:49.636131 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf04d35e-fa03-4b9d-a74d-f1185d85c463 service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:38:49.636482 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf04d35e-fa03-4b9d-a74d-f1185d85c463 service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:38:49.636830 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf04d35e-fa03-4b9d-a74d-f1185d85c463 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:38:49.641100 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-cf04d35e-fa03-4b9d-a74d-f1185d85c463 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:38:49.715920 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf04d35e-fa03-4b9d-a74d-f1185d85c463 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:38:49.717565 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf04d35e-fa03-4b9d-a74d-f1185d85c463 service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:38:49.729810 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1185/2381] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:38:49 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:38:52.867956 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71f92c0-d1c3-46c0-9eb4-f2f649977265 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:38:52.868504 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71f92c0-d1c3-46c0-9eb4-f2f649977265 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:38:52.868504 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71f92c0-d1c3-46c0-9eb4-f2f649977265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:38:52.887885 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-b71f92c0-d1c3-46c0-9eb4-f2f649977265 None None] MFA Rules not processed for user `5cb73f6047dd42aaba23072f51972e9f`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:38:52.889263 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-b71f92c0-d1c3-46c0-9eb4-f2f649977265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:38:52.896961 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1197/2382] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:38:52 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:38:53.190979 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf04d35e-fa03-4b9d-a74d-f1185d85c463 service nova] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:38:53.193270 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-cf04d35e-fa03-4b9d-a74d-f1185d85c463 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:38:53.208623 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf04d35e-fa03-4b9d-a74d-f1185d85c463 service nova] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:38:53.214499 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eaa7b0-25db-4b87-8eca-a8e520e3aba0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87adc1d5bc240e99f95f6595b350be4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:38:53.215130 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eaa7b0-25db-4b87-8eca-a8e520e3aba0 service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:38:53.215473 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eaa7b0-25db-4b87-8eca-a8e520e3aba0 service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:38:53.215757 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eaa7b0-25db-4b87-8eca-a8e520e3aba0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:38:53.217679 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-d8eaa7b0-25db-4b87-8eca-a8e520e3aba0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:38:53.230495 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8eaa7b0-25db-4b87-8eca-a8e520e3aba0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:38:53.232070 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8eaa7b0-25db-4b87-8eca-a8e520e3aba0 service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:38:53.234920 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71f92c0-d1c3-46c0-9eb4-f2f649977265 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:38:53.236311 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-b71f92c0-d1c3-46c0-9eb4-f2f649977265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:38:53.243500 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1186/2383] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:38:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:38:53.298795 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71f92c0-d1c3-46c0-9eb4-f2f649977265 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:38:53.302948 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b40648-c6ad-4b04-9c2d-f3019fa5a68c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:38:53.303569 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b40648-c6ad-4b04-9c2d-f3019fa5a68c service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:38:53.303854 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b40648-c6ad-4b04-9c2d-f3019fa5a68c service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:38:53.304130 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b40648-c6ad-4b04-9c2d-f3019fa5a68c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:38:53.305991 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-e4b40648-c6ad-4b04-9c2d-f3019fa5a68c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:38:53.315998 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b40648-c6ad-4b04-9c2d-f3019fa5a68c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:38:53.318105 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b40648-c6ad-4b04-9c2d-f3019fa5a68c service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:38:53.327353 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1198/2384] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:38:53 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 01:38:55.680114 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c6a814-15a1-42ae-9200-dbbe363e7142 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:38:55.680627 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c6a814-15a1-42ae-9200-dbbe363e7142 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:38:55.680734 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c6a814-15a1-42ae-9200-dbbe363e7142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:38:55.700958 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-09c6a814-15a1-42ae-9200-dbbe363e7142 None None] MFA Rules not processed for user `8e86f38590824894a3b960e6774ff8a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:38:55.702192 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-09c6a814-15a1-42ae-9200-dbbe363e7142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:38:55.710557 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1187/2385] 10.222.0.26 () {58 vars in 967 bytes} [Sat Aug 6 01:38:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:38:55.728438 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b40648-c6ad-4b04-9c2d-f3019fa5a68c service neutron] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:38:55.740671 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b40648-c6ad-4b04-9c2d-f3019fa5a68c service neutron] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:38:55.745885 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09252c8c-7e42-47da-96c3-2aad38b32977 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b23f4478f49471fba6f43e6e4be99da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:38:55.746517 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09252c8c-7e42-47da-96c3-2aad38b32977 service swift] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:38:55.746798 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09252c8c-7e42-47da-96c3-2aad38b32977 service swift] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:38:55.747069 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09252c8c-7e42-47da-96c3-2aad38b32977 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:38:55.755193 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09252c8c-7e42-47da-96c3-2aad38b32977 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:38:55.756766 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09252c8c-7e42-47da-96c3-2aad38b32977 service swift] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:38:55.760211 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:38:55.766145 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1199/2386] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Aug 6 01:38:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 06 01:38:57.674497 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3109a3-aa34-429e-b7b4-5ba3ec56322c None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:38:57.674963 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3109a3-aa34-429e-b7b4-5ba3ec56322c None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:38:57.675079 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3109a3-aa34-429e-b7b4-5ba3ec56322c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:38:57.701636 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-8c3109a3-aa34-429e-b7b4-5ba3ec56322c None None] MFA Rules not processed for user `8e86f38590824894a3b960e6774ff8a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:38:57.703424 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-8c3109a3-aa34-429e-b7b4-5ba3ec56322c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:38:57.713539 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1188/2387] 10.222.0.26 () {58 vars in 1009 bytes} [Sat Aug 6 01:38:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:38:57.728342 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09252c8c-7e42-47da-96c3-2aad38b32977 service swift] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:38:57.740277 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09252c8c-7e42-47da-96c3-2aad38b32977 service swift] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:38:57.747843 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5340ca80-8657-45ae-8c50-1302b0c36982 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b23f4478f49471fba6f43e6e4be99da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:38:57.748748 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5340ca80-8657-45ae-8c50-1302b0c36982 service swift] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:38:57.749696 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5340ca80-8657-45ae-8c50-1302b0c36982 service swift] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:38:57.750043 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5340ca80-8657-45ae-8c50-1302b0c36982 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:38:57.762462 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5340ca80-8657-45ae-8c50-1302b0c36982 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:38:57.764838 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5340ca80-8657-45ae-8c50-1302b0c36982 service swift] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:38:57.782497 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1200/2388] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Aug 6 01:38:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 06 01:38:58.841042 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3109a3-aa34-429e-b7b4-5ba3ec56322c None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:38:58.860119 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3109a3-aa34-429e-b7b4-5ba3ec56322c None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:38:58.867212 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e229f9c-6352-4480-9e49-4505184c11dc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87adc1d5bc240e99f95f6595b350be4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:38:58.868381 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e229f9c-6352-4480-9e49-4505184c11dc service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:38:58.868715 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e229f9c-6352-4480-9e49-4505184c11dc service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:38:58.868946 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e229f9c-6352-4480-9e49-4505184c11dc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:38:58.871064 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-1e229f9c-6352-4480-9e49-4505184c11dc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:38:58.885263 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e229f9c-6352-4480-9e49-4505184c11dc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:38:58.887176 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e229f9c-6352-4480-9e49-4505184c11dc service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:38:58.903518 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1189/2389] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:38:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:39:07.772491 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=7 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:39:08.899024 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=47 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:39:27.158071 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5340ca80-8657-45ae-8c50-1302b0c36982 service swift] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:39:27.170381 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5340ca80-8657-45ae-8c50-1302b0c36982 service swift] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:39:27.175994 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74fb0b0-cd24-4e85-a572-ca5cbbda73d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87adc1d5bc240e99f95f6595b350be4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:39:27.176662 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74fb0b0-cd24-4e85-a572-ca5cbbda73d0 service nova] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:39:27.176944 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74fb0b0-cd24-4e85-a572-ca5cbbda73d0 service nova] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:39:27.177254 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74fb0b0-cd24-4e85-a572-ca5cbbda73d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:39:27.179786 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-d74fb0b0-cd24-4e85-a572-ca5cbbda73d0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:39:27.236910 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74fb0b0-cd24-4e85-a572-ca5cbbda73d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:39:27.238428 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74fb0b0-cd24-4e85-a572-ca5cbbda73d0 service nova] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:39:27.248541 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1201/2390] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:39:27 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:39:32.300521 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e229f9c-6352-4480-9e49-4505184c11dc service nova] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:39:32.309970 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e229f9c-6352-4480-9e49-4505184c11dc service nova] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:39:32.313609 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bb29d0-0a3d-43d8-b59b-e0710d1928c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:39:32.314280 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bb29d0-0a3d-43d8-b59b-e0710d1928c8 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:39:32.314569 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bb29d0-0a3d-43d8-b59b-e0710d1928c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:39:32.314849 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bb29d0-0a3d-43d8-b59b-e0710d1928c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:39:32.323852 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bb29d0-0a3d-43d8-b59b-e0710d1928c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:39:32.325311 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bb29d0-0a3d-43d8-b59b-e0710d1928c8 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:39:32.334771 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1190/2391] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:39:32 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:39:35.756121 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e724a960-0f0d-4f11-a111-5bedba676d46 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:39:35.757937 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e724a960-0f0d-4f11-a111-5bedba676d46 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:39:35.758398 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e724a960-0f0d-4f11-a111-5bedba676d46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:39:35.779170 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-e724a960-0f0d-4f11-a111-5bedba676d46 None None] MFA Rules not processed for user `37c704ce557c4f309cf00ef32646235f`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:39:35.780755 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-e724a960-0f0d-4f11-a111-5bedba676d46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:39:35.787737 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1202/2392] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:39:35 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:39:42.330451 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=6 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:39:45.772399 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=31 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:40:05.803621 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bb29d0-0a3d-43d8-b59b-e0710d1928c8 service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:40:05.815872 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bb29d0-0a3d-43d8-b59b-e0710d1928c8 service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:40:05.821476 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71511685-ea73-45bc-b5f2-d422194637f7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87adc1d5bc240e99f95f6595b350be4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:40:05.822088 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71511685-ea73-45bc-b5f2-d422194637f7 service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:40:05.822372 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71511685-ea73-45bc-b5f2-d422194637f7 service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:40:05.822643 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71511685-ea73-45bc-b5f2-d422194637f7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:40:05.824366 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-71511685-ea73-45bc-b5f2-d422194637f7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:40:05.888586 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71511685-ea73-45bc-b5f2-d422194637f7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:40:05.890237 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71511685-ea73-45bc-b5f2-d422194637f7 service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:40:05.900230 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1191/2393] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:40:05 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:40:09.921613 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cbb4ed-325e-4ec3-969a-e06eea0e65e3 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:40:09.922227 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cbb4ed-325e-4ec3-969a-e06eea0e65e3 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:40:09.922227 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cbb4ed-325e-4ec3-969a-e06eea0e65e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:40:09.942078 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-d6cbb4ed-325e-4ec3-969a-e06eea0e65e3 None None] MFA Rules not processed for user `42ade772e11a4725928ab12d3bccc4a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:40:09.943289 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-d6cbb4ed-325e-4ec3-969a-e06eea0e65e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:40:09.999943 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1203/2394] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:40:09 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:40:15.895743 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=28 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:40:19.992166 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=25 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:40:22.328206 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5adf96-d7ed-465d-bb08-d4c14777e3db None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:40:22.328770 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5adf96-d7ed-465d-bb08-d4c14777e3db None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:40:22.329156 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5adf96-d7ed-465d-bb08-d4c14777e3db None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:40:22.330609 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1192/2395] 10.222.0.26 () {54 vars in 921 bytes} [Sat Aug 6 01:40:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:40:22.338055 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9600b6b2-5928-48cc-b92b-4f3f7eb0f212 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:40:22.338530 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9600b6b2-5928-48cc-b92b-4f3f7eb0f212 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:40:22.338656 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9600b6b2-5928-48cc-b92b-4f3f7eb0f212 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:40:22.358479 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-9600b6b2-5928-48cc-b92b-4f3f7eb0f212 None None] MFA Rules not processed for user `4ac0e31c84b4445fa167bbe4954066c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:40:22.359692 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-9600b6b2-5928-48cc-b92b-4f3f7eb0f212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:40:22.366460 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1204/2396] 10.222.0.26 () {58 vars in 1018 bytes} [Sat Aug 6 01:40:22 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:40:22.380924 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5adf96-d7ed-465d-bb08-d4c14777e3db None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:40:22.401080 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5adf96-d7ed-465d-bb08-d4c14777e3db None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:40:22.409235 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7587c8-97ab-49cd-9271-317d9638db39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:40:22.410370 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7587c8-97ab-49cd-9271-317d9638db39 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:40:22.410662 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7587c8-97ab-49cd-9271-317d9638db39 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:40:22.410939 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7587c8-97ab-49cd-9271-317d9638db39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:40:22.419375 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7587c8-97ab-49cd-9271-317d9638db39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:40:22.420859 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7587c8-97ab-49cd-9271-317d9638db39 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:40:22.429914 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1193/2397] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:40:22 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:40:22.513620 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710530af-b535-4318-a5f1-99d03664f996 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:40:22.513914 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710530af-b535-4318-a5f1-99d03664f996 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:40:22.514197 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710530af-b535-4318-a5f1-99d03664f996 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:40:22.515556 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1205/2398] 10.222.0.26 () {54 vars in 921 bytes} [Sat Aug 6 01:40:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:40:22.520963 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c06aa9-4ddc-44fd-8e6e-182b652fed38 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:40:22.521280 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c06aa9-4ddc-44fd-8e6e-182b652fed38 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:40:22.521563 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c06aa9-4ddc-44fd-8e6e-182b652fed38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:40:22.541928 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-48c06aa9-4ddc-44fd-8e6e-182b652fed38 None None] MFA Rules not processed for user `b87adc1d5bc240e99f95f6595b350be4`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:40:22.543123 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-48c06aa9-4ddc-44fd-8e6e-182b652fed38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:40:22.552401 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1194/2399] 10.222.0.26 () {58 vars in 1018 bytes} [Sat Aug 6 01:40:22 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:40:22.562909 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710530af-b535-4318-a5f1-99d03664f996 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:40:22.574349 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710530af-b535-4318-a5f1-99d03664f996 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:40:22.577941 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b588fb-1a0e-4bc4-883b-844b16d75594 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:40:22.578542 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b588fb-1a0e-4bc4-883b-844b16d75594 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:40:22.578824 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b588fb-1a0e-4bc4-883b-844b16d75594 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:40:22.579098 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b588fb-1a0e-4bc4-883b-844b16d75594 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:40:22.589670 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b588fb-1a0e-4bc4-883b-844b16d75594 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:40:22.591089 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b588fb-1a0e-4bc4-883b-844b16d75594 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:40:22.601637 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1206/2400] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:40:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:40:24.177952 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c06aa9-4ddc-44fd-8e6e-182b652fed38 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:40:24.190155 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c06aa9-4ddc-44fd-8e6e-182b652fed38 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:40:24.195903 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdf7dd2-3908-4cc6-bb40-2908cc388c32 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87adc1d5bc240e99f95f6595b350be4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:40:24.196530 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdf7dd2-3908-4cc6-bb40-2908cc388c32 service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:40:24.196819 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdf7dd2-3908-4cc6-bb40-2908cc388c32 service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:40:24.197132 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdf7dd2-3908-4cc6-bb40-2908cc388c32 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:40:24.199261 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-5cdf7dd2-3908-4cc6-bb40-2908cc388c32 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:40:24.212823 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdf7dd2-3908-4cc6-bb40-2908cc388c32 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:40:24.214449 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdf7dd2-3908-4cc6-bb40-2908cc388c32 service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:40:24.224853 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1195/2401] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:40:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:40:32.597016 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=9 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:40:34.220469 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=15 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:41:06.736912 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b588fb-1a0e-4bc4-883b-844b16d75594 service neutron] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:41:06.762335 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b588fb-1a0e-4bc4-883b-844b16d75594 service neutron] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:41:06.766211 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aa90ca-1569-4785-bfff-61a77d11e7e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:41:06.766829 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aa90ca-1569-4785-bfff-61a77d11e7e6 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:41:06.767112 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aa90ca-1569-4785-bfff-61a77d11e7e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:41:06.767471 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aa90ca-1569-4785-bfff-61a77d11e7e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:41:06.769292 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-37aa90ca-1569-4785-bfff-61a77d11e7e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:41:06.907368 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37aa90ca-1569-4785-bfff-61a77d11e7e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:41:06.908944 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37aa90ca-1569-4785-bfff-61a77d11e7e6 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:41:06.929636 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1207/2402] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:41:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:41:08.304079 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ab2d53-7aec-4300-8cd8-a3e700c6e451 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:41:08.304578 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ab2d53-7aec-4300-8cd8-a3e700c6e451 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:41:08.304866 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ab2d53-7aec-4300-8cd8-a3e700c6e451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:41:08.326010 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-90ab2d53-7aec-4300-8cd8-a3e700c6e451 None None] MFA Rules not processed for user `2455ea8abba547669fe22d2065a726c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:41:08.327228 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-90ab2d53-7aec-4300-8cd8-a3e700c6e451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:41:08.334157 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1196/2403] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:41:08 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:41:10.454395 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aa90ca-1569-4785-bfff-61a77d11e7e6 service neutron] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:41:10.455903 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-37aa90ca-1569-4785-bfff-61a77d11e7e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:41:10.525578 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ab2d53-7aec-4300-8cd8-a3e700c6e451 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:41:10.527930 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-90ab2d53-7aec-4300-8cd8-a3e700c6e451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:41:10.553290 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ab2d53-7aec-4300-8cd8-a3e700c6e451 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:41:10.558263 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89776b54-fb18-4576-b945-b63501ab3174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:41:10.559211 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89776b54-fb18-4576-b945-b63501ab3174 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:41:10.559673 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89776b54-fb18-4576-b945-b63501ab3174 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:41:10.559956 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89776b54-fb18-4576-b945-b63501ab3174 admin admin] PATH_INFO: `/v3/users/37c704ce557c4f309cf00ef32646235f` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:41:10.561335 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89776b54-fb18-4576-b945-b63501ab3174 admin admin] RBAC: Authorizing `identity:delete_user(user_id=37c704ce557c4f309cf00ef32646235f)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:41:10.563283 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89776b54-fb18-4576-b945-b63501ab3174 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:41:10.582851 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:41:10.594896 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aa90ca-1569-4785-bfff-61a77d11e7e6 service neutron] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:41:10.601316 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f05cb71-fcb5-4273-8cda-fda213af9e26 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa3b93ddd9464a2cba724d67b56b477a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:41:10.601937 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f05cb71-fcb5-4273-8cda-fda213af9e26 service placement] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:41:10.603082 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f05cb71-fcb5-4273-8cda-fda213af9e26 service placement] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:41:10.603511 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f05cb71-fcb5-4273-8cda-fda213af9e26 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:41:10.605817 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-6f05cb71-fcb5-4273-8cda-fda213af9e26 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:41:10.619796 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f05cb71-fcb5-4273-8cda-fda213af9e26 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:41:10.622166 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f05cb71-fcb5-4273-8cda-fda213af9e26 service placement] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:41:10.623662 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-89776b54-fb18-4576-b945-b63501ab3174 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37c704ce557c4f309cf00ef32646235f'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:41:10.624408 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-89776b54-fb18-4576-b945-b63501ab3174 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37c704ce557c4f309cf00ef32646235f'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:41:10.625097 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-89776b54-fb18-4576-b945-b63501ab3174 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37c704ce557c4f309cf00ef32646235f'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:41:10.633325 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1208/2404] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:41:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 01:41:10.642386 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-89776b54-fb18-4576-b945-b63501ab3174 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37c704ce557c4f309cf00ef32646235f'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:41:10.654236 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-89776b54-fb18-4576-b945-b63501ab3174 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37c704ce557c4f309cf00ef32646235f'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:41:10.671366 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1197/2405] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Aug 6 01:41:10 2022] DELETE /identity/v3/users/37c704ce557c4f309cf00ef32646235f => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:41:10.850100 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f05cb71-fcb5-4273-8cda-fda213af9e26 service placement] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:41:10.867246 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f05cb71-fcb5-4273-8cda-fda213af9e26 service placement] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:41:10.929993 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2597bf-6a6d-403b-b71b-482b64b5f242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:41:10.931386 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2597bf-6a6d-403b-b71b-482b64b5f242 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:41:10.931742 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2597bf-6a6d-403b-b71b-482b64b5f242 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:41:10.932090 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2597bf-6a6d-403b-b71b-482b64b5f242 admin admin] PATH_INFO: `/v3/projects/5be6efff0f0d482180e74f1cd075b592` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:41:10.933384 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2597bf-6a6d-403b-b71b-482b64b5f242 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5be6efff0f0d482180e74f1cd075b592)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:41:10.935306 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2597bf-6a6d-403b-b71b-482b64b5f242 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:41:10.989819 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-9b2597bf-6a6d-403b-b71b-482b64b5f242 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5be6efff0f0d482180e74f1cd075b592', 'request_id': 'req-9b2597bf-6a6d-403b-b71b-482b64b5f242'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:41:10.998191 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-9b2597bf-6a6d-403b-b71b-482b64b5f242 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5be6efff0f0d482180e74f1cd075b592', 'request_id': 'req-9b2597bf-6a6d-403b-b71b-482b64b5f242'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:41:11.004825 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-9b2597bf-6a6d-403b-b71b-482b64b5f242 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5be6efff0f0d482180e74f1cd075b592', 'request_id': 'req-9b2597bf-6a6d-403b-b71b-482b64b5f242'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:41:11.023019 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-9b2597bf-6a6d-403b-b71b-482b64b5f242 admin admin] The token cache is being invalidate because project 5be6efff0f0d482180e74f1cd075b592 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:41:11.023563 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-9b2597bf-6a6d-403b-b71b-482b64b5f242 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:41:11.025351 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-9b2597bf-6a6d-403b-b71b-482b64b5f242 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:41:11.027607 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1209/2406] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Aug 6 01:41:10 2022] DELETE /identity/v3/projects/5be6efff0f0d482180e74f1cd075b592 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:41:11.032521 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad408d1-eef5-49eb-a9bc-aa1f362943f4 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:41:11.032829 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad408d1-eef5-49eb-a9bc-aa1f362943f4 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:41:11.033133 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad408d1-eef5-49eb-a9bc-aa1f362943f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:41:11.096976 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-fad408d1-eef5-49eb-a9bc-aa1f362943f4 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:41:11.098313 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-fad408d1-eef5-49eb-a9bc-aa1f362943f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:41:11.166395 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1198/2407] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:41:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:41:11.174804 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c32417-6e24-473d-94e8-a25ea82e1d8c None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:41:11.175117 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c32417-6e24-473d-94e8-a25ea82e1d8c None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:41:11.176015 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c32417-6e24-473d-94e8-a25ea82e1d8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:41:11.198237 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-13c32417-6e24-473d-94e8-a25ea82e1d8c None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:41:11.199475 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-13c32417-6e24-473d-94e8-a25ea82e1d8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:41:11.207475 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1210/2408] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:41:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:41:11.212658 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad408d1-eef5-49eb-a9bc-aa1f362943f4 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:41:11.222393 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad408d1-eef5-49eb-a9bc-aa1f362943f4 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:41:11.227045 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc3c0da-3786-49bb-bc81-1f8fd6faf2be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:41:11.227718 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc3c0da-3786-49bb-bc81-1f8fd6faf2be admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:41:11.228003 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc3c0da-3786-49bb-bc81-1f8fd6faf2be admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:41:11.228283 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc3c0da-3786-49bb-bc81-1f8fd6faf2be admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:41:11.228992 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc3c0da-3786-49bb-bc81-1f8fd6faf2be admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:41:11.229327 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc3c0da-3786-49bb-bc81-1f8fd6faf2be admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:41:11.231221 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc3c0da-3786-49bb-bc81-1f8fd6faf2be admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:41:11.243203 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1199/2409] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Aug 6 01:41:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:41:11.247946 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c32417-6e24-473d-94e8-a25ea82e1d8c None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:41:11.257347 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c32417-6e24-473d-94e8-a25ea82e1d8c None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:41:11.263552 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0038a6a-887d-4125-954a-b50f70e1aafc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:41:11.276131 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0038a6a-887d-4125-954a-b50f70e1aafc admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:41:11.276515 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0038a6a-887d-4125-954a-b50f70e1aafc admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:41:11.276873 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0038a6a-887d-4125-954a-b50f70e1aafc admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:41:11.277683 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0038a6a-887d-4125-954a-b50f70e1aafc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:41:11.280176 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0038a6a-887d-4125-954a-b50f70e1aafc admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:41:11.325543 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1211/2410] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Aug 6 01:41:11 2022] POST /identity/v3/projects => generated 385 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:41:11.329582 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc3c0da-3786-49bb-bc81-1f8fd6faf2be admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:41:11.340466 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc3c0da-3786-49bb-bc81-1f8fd6faf2be admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:41:11.358919 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:41:11.411438 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee9296c-699b-417c-9ad9-f05fccb38c36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:41:11.412134 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee9296c-699b-417c-9ad9-f05fccb38c36 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:41:11.412424 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee9296c-699b-417c-9ad9-f05fccb38c36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:41:11.412705 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee9296c-699b-417c-9ad9-f05fccb38c36 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:41:11.413933 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee9296c-699b-417c-9ad9-f05fccb38c36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:41:11.415372 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee9296c-699b-417c-9ad9-f05fccb38c36 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:41:11.437861 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1200/2411] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:41:11 2022] POST /identity/v3/users => generated 293 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:41:11.441723 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0038a6a-887d-4125-954a-b50f70e1aafc admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:41:11.453671 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0038a6a-887d-4125-954a-b50f70e1aafc admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:41:11.468458 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff075419-ecec-4be9-b453-d83936ad5aed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:41:11.469897 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff075419-ecec-4be9-b453-d83936ad5aed admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:41:11.470345 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff075419-ecec-4be9-b453-d83936ad5aed admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:41:11.470625 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff075419-ecec-4be9-b453-d83936ad5aed admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:41:11.471340 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff075419-ecec-4be9-b453-d83936ad5aed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:41:11.472970 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff075419-ecec-4be9-b453-d83936ad5aed admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:41:11.484109 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1212/2412] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:41:11 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 01:41:11.488164 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee9296c-699b-417c-9ad9-f05fccb38c36 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:41:11.503858 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee9296c-699b-417c-9ad9-f05fccb38c36 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:41:11.532173 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363a10f1-68bf-42dd-af8e-aef616d8d9b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:41:11.532904 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363a10f1-68bf-42dd-af8e-aef616d8d9b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:41:11.533428 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363a10f1-68bf-42dd-af8e-aef616d8d9b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:41:11.533779 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363a10f1-68bf-42dd-af8e-aef616d8d9b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:41:11.534497 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363a10f1-68bf-42dd-af8e-aef616d8d9b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:41:11.536182 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363a10f1-68bf-42dd-af8e-aef616d8d9b5 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:41:11.548329 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1201/2413] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:41:11 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 01:41:11.552574 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff075419-ecec-4be9-b453-d83936ad5aed admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:41:11.566355 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff075419-ecec-4be9-b453-d83936ad5aed admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:41:11.571713 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c330c9fd-0bf5-409f-bc0f-2cb8cbf6ad5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:41:11.572707 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c330c9fd-0bf5-409f-bc0f-2cb8cbf6ad5b admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:41:11.573104 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c330c9fd-0bf5-409f-bc0f-2cb8cbf6ad5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:41:11.573460 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c330c9fd-0bf5-409f-bc0f-2cb8cbf6ad5b admin admin] PATH_INFO: `/v3/projects/7906e11569cc47778d8ed1c6b371ae4d/users/d2235317fe8d4d7ebae7bb5b80dfd7e6/roles/61b3b93d37a047e4bb8369b46fe20db9` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:41:11.589359 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c330c9fd-0bf5-409f-bc0f-2cb8cbf6ad5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7906e11569cc47778d8ed1c6b371ae4d, user_id=d2235317fe8d4d7ebae7bb5b80dfd7e6, role_id=61b3b93d37a047e4bb8369b46fe20db9)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:41:11.592246 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c330c9fd-0bf5-409f-bc0f-2cb8cbf6ad5b admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:41:11.606691 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1213/2414] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:41:11 2022] PUT /identity/v3/projects/7906e11569cc47778d8ed1c6b371ae4d/users/d2235317fe8d4d7ebae7bb5b80dfd7e6/roles/61b3b93d37a047e4bb8369b46fe20db9 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:41:11.612128 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9390e128-cc09-4567-8d22-a5f2f7d6b76b None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:41:11.612363 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9390e128-cc09-4567-8d22-a5f2f7d6b76b None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:41:11.612669 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9390e128-cc09-4567-8d22-a5f2f7d6b76b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:41:11.632403 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-9390e128-cc09-4567-8d22-a5f2f7d6b76b None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:41:11.634293 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-9390e128-cc09-4567-8d22-a5f2f7d6b76b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:41:11.699048 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1202/2415] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:41:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:41:11.708874 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c330c9fd-0bf5-409f-bc0f-2cb8cbf6ad5b admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:41:11.710595 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-c330c9fd-0bf5-409f-bc0f-2cb8cbf6ad5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:41:11.728814 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:41:11.795830 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c330c9fd-0bf5-409f-bc0f-2cb8cbf6ad5b admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:41:11.799866 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d538949e-a138-4e9f-ac31-28f34d911bac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:41:11.800500 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d538949e-a138-4e9f-ac31-28f34d911bac service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:41:11.800783 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d538949e-a138-4e9f-ac31-28f34d911bac service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:41:11.801078 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d538949e-a138-4e9f-ac31-28f34d911bac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:41:11.816122 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d538949e-a138-4e9f-ac31-28f34d911bac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:41:11.817685 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d538949e-a138-4e9f-ac31-28f34d911bac service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:41:11.830853 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1214/2416] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:41:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:41:15.937554 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f86ada-1c71-47ff-83c1-d5e04fc9b767 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:41:15.938099 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f86ada-1c71-47ff-83c1-d5e04fc9b767 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:41:15.938099 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f86ada-1c71-47ff-83c1-d5e04fc9b767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:41:15.970182 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-b9f86ada-1c71-47ff-83c1-d5e04fc9b767 None None] MFA Rules not processed for user `d2235317fe8d4d7ebae7bb5b80dfd7e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:41:15.972352 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-b9f86ada-1c71-47ff-83c1-d5e04fc9b767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:41:16.050759 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1203/2417] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:41:15 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:41:16.066113 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d538949e-a138-4e9f-ac31-28f34d911bac service neutron] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:41:16.067552 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-d538949e-a138-4e9f-ac31-28f34d911bac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:41:16.184307 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d538949e-a138-4e9f-ac31-28f34d911bac service neutron] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:41:16.191101 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748992cf-fdec-4ee7-86fa-468be5079f05 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87adc1d5bc240e99f95f6595b350be4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:41:16.191777 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748992cf-fdec-4ee7-86fa-468be5079f05 service nova] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:41:16.192066 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748992cf-fdec-4ee7-86fa-468be5079f05 service nova] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:41:16.192348 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748992cf-fdec-4ee7-86fa-468be5079f05 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:41:16.204173 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748992cf-fdec-4ee7-86fa-468be5079f05 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:41:16.205942 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748992cf-fdec-4ee7-86fa-468be5079f05 service nova] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:41:16.216380 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1215/2418] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:41:16 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:41:17.518253 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238fa522-30be-4788-837e-18fe3ab51e54 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:41:17.518700 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238fa522-30be-4788-837e-18fe3ab51e54 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:41:17.518972 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238fa522-30be-4788-837e-18fe3ab51e54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:41:17.541593 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-238fa522-30be-4788-837e-18fe3ab51e54 None None] MFA Rules not processed for user `d2235317fe8d4d7ebae7bb5b80dfd7e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:41:17.542909 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-238fa522-30be-4788-837e-18fe3ab51e54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:41:17.551823 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1204/2419] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:41:17 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:41:26.211282 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=75 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:41:27.535623 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=86,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:42:00.382515 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748992cf-fdec-4ee7-86fa-468be5079f05 service nova] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:42:00.384816 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-748992cf-fdec-4ee7-86fa-468be5079f05 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:42:00.457022 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748992cf-fdec-4ee7-86fa-468be5079f05 service nova] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:42:00.462304 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535c4ca5-1ac4-4589-ac46-dae829083695 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa3b93ddd9464a2cba724d67b56b477a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:42:00.462946 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535c4ca5-1ac4-4589-ac46-dae829083695 service placement] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:42:00.463227 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535c4ca5-1ac4-4589-ac46-dae829083695 service placement] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:42:00.463503 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535c4ca5-1ac4-4589-ac46-dae829083695 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:42:00.465319 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-535c4ca5-1ac4-4589-ac46-dae829083695 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:42:00.476506 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535c4ca5-1ac4-4589-ac46-dae829083695 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:42:00.478241 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535c4ca5-1ac4-4589-ac46-dae829083695 service placement] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:42:00.487877 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1216/2420] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:42:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:42:06.948255 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27b05d3-59fd-42ca-ab8b-b8e6ac5da955 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:42:06.949348 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27b05d3-59fd-42ca-ab8b-b8e6ac5da955 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:42:06.949638 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27b05d3-59fd-42ca-ab8b-b8e6ac5da955 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:42:06.985557 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-a27b05d3-59fd-42ca-ab8b-b8e6ac5da955 None None] MFA Rules not processed for user `8e86f38590824894a3b960e6774ff8a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:42:06.986577 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-a27b05d3-59fd-42ca-ab8b-b8e6ac5da955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:42:07.053216 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1205/2421] 10.222.0.26 () {58 vars in 1009 bytes} [Sat Aug 6 01:42:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:42:07.102033 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535c4ca5-1ac4-4589-ac46-dae829083695 service placement] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:42:07.103517 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-535c4ca5-1ac4-4589-ac46-dae829083695 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:42:07.183349 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535c4ca5-1ac4-4589-ac46-dae829083695 service placement] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:42:07.189493 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bcefd8-359e-4ab1-8239-19af2ae9dff2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b23f4478f49471fba6f43e6e4be99da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:42:07.190333 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bcefd8-359e-4ab1-8239-19af2ae9dff2 service swift] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:42:07.190799 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bcefd8-359e-4ab1-8239-19af2ae9dff2 service swift] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:42:07.191246 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bcefd8-359e-4ab1-8239-19af2ae9dff2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:42:07.201168 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bcefd8-359e-4ab1-8239-19af2ae9dff2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:42:07.203033 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bcefd8-359e-4ab1-8239-19af2ae9dff2 service swift] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:42:07.213443 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1217/2422] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Aug 6 01:42:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 06 01:42:15.105253 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6e6185-b2db-45e5-9365-6a5c5328ad5b None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:42:15.105833 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6e6185-b2db-45e5-9365-6a5c5328ad5b None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:42:15.105833 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6e6185-b2db-45e5-9365-6a5c5328ad5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:42:15.126301 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-cd6e6185-b2db-45e5-9365-6a5c5328ad5b None None] MFA Rules not processed for user `8e86f38590824894a3b960e6774ff8a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:42:15.127766 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-cd6e6185-b2db-45e5-9365-6a5c5328ad5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:42:15.136542 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1206/2423] 10.222.0.26 () {58 vars in 967 bytes} [Sat Aug 6 01:42:15 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:42:15.153136 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bcefd8-359e-4ab1-8239-19af2ae9dff2 service swift] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:42:15.165189 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bcefd8-359e-4ab1-8239-19af2ae9dff2 service swift] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:42:15.170855 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb199a1-5a41-4e4f-a979-6f0efb1e1908 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b23f4478f49471fba6f43e6e4be99da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:42:15.171587 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb199a1-5a41-4e4f-a979-6f0efb1e1908 service swift] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:42:15.171964 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb199a1-5a41-4e4f-a979-6f0efb1e1908 service swift] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:42:15.172440 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb199a1-5a41-4e4f-a979-6f0efb1e1908 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:42:15.184495 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb199a1-5a41-4e4f-a979-6f0efb1e1908 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:42:15.186224 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb199a1-5a41-4e4f-a979-6f0efb1e1908 service swift] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:42:15.198834 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1218/2424] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Aug 6 01:42:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 06 01:42:23.826159 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6e6185-b2db-45e5-9365-6a5c5328ad5b None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:42:23.827863 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-cd6e6185-b2db-45e5-9365-6a5c5328ad5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:42:23.848093 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6e6185-b2db-45e5-9365-6a5c5328ad5b None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:42:23.851842 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db67c8d-5b2a-404f-966b-854f54ca8b3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:42:23.852718 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db67c8d-5b2a-404f-966b-854f54ca8b3c service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:42:23.852952 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db67c8d-5b2a-404f-966b-854f54ca8b3c service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:42:23.853288 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db67c8d-5b2a-404f-966b-854f54ca8b3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:42:23.855113 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-8db67c8d-5b2a-404f-966b-854f54ca8b3c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:42:23.873430 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db67c8d-5b2a-404f-966b-854f54ca8b3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:42:23.875546 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db67c8d-5b2a-404f-966b-854f54ca8b3c service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:42:23.888247 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1207/2425] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:42:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:42:24.363583 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb199a1-5a41-4e4f-a979-6f0efb1e1908 service swift] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:42:24.365170 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-bfb199a1-5a41-4e4f-a979-6f0efb1e1908 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:42:24.380721 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb199a1-5a41-4e4f-a979-6f0efb1e1908 service swift] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:42:24.386672 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1f5e2a-c671-4af7-a6c2-5b0f65430294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:42:24.387390 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1f5e2a-c671-4af7-a6c2-5b0f65430294 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:42:24.387749 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1f5e2a-c671-4af7-a6c2-5b0f65430294 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:42:24.388096 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1f5e2a-c671-4af7-a6c2-5b0f65430294 admin admin] PATH_INFO: `/v3/users/5cb73f6047dd42aaba23072f51972e9f` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:42:24.389431 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1f5e2a-c671-4af7-a6c2-5b0f65430294 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5cb73f6047dd42aaba23072f51972e9f)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:42:24.391558 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1f5e2a-c671-4af7-a6c2-5b0f65430294 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:42:24.447275 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-ac1f5e2a-c671-4af7-a6c2-5b0f65430294 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5cb73f6047dd42aaba23072f51972e9f'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:42:24.454633 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-ac1f5e2a-c671-4af7-a6c2-5b0f65430294 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5cb73f6047dd42aaba23072f51972e9f'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:42:24.455264 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-ac1f5e2a-c671-4af7-a6c2-5b0f65430294 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5cb73f6047dd42aaba23072f51972e9f'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:42:24.455877 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-ac1f5e2a-c671-4af7-a6c2-5b0f65430294 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5cb73f6047dd42aaba23072f51972e9f'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:42:24.460314 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:42:24.470140 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-ac1f5e2a-c671-4af7-a6c2-5b0f65430294 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5cb73f6047dd42aaba23072f51972e9f'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:42:24.484344 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1219/2426] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Aug 6 01:42:24 2022] DELETE /identity/v3/users/5cb73f6047dd42aaba23072f51972e9f => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:42:24.672917 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db67c8d-5b2a-404f-966b-854f54ca8b3c service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:42:24.683512 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db67c8d-5b2a-404f-966b-854f54ca8b3c service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:42:24.746691 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faa779b-c98c-4338-b216-12b15dc91f04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:42:24.747347 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faa779b-c98c-4338-b216-12b15dc91f04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:42:24.747637 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faa779b-c98c-4338-b216-12b15dc91f04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:42:24.747905 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faa779b-c98c-4338-b216-12b15dc91f04 admin admin] PATH_INFO: `/v3/projects/4db9537bd65748f6b43061f3530f6891` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:42:24.749077 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5faa779b-c98c-4338-b216-12b15dc91f04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4db9537bd65748f6b43061f3530f6891)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:42:24.750992 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5faa779b-c98c-4338-b216-12b15dc91f04 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:42:24.815037 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-5faa779b-c98c-4338-b216-12b15dc91f04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4db9537bd65748f6b43061f3530f6891', 'request_id': 'req-5faa779b-c98c-4338-b216-12b15dc91f04'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:42:24.822826 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-5faa779b-c98c-4338-b216-12b15dc91f04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4db9537bd65748f6b43061f3530f6891', 'request_id': 'req-5faa779b-c98c-4338-b216-12b15dc91f04'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:42:24.831919 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-5faa779b-c98c-4338-b216-12b15dc91f04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4db9537bd65748f6b43061f3530f6891', 'request_id': 'req-5faa779b-c98c-4338-b216-12b15dc91f04'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:42:24.846362 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-5faa779b-c98c-4338-b216-12b15dc91f04 admin admin] The token cache is being invalidate because project 4db9537bd65748f6b43061f3530f6891 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:42:24.847001 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-5faa779b-c98c-4338-b216-12b15dc91f04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:42:24.848026 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-5faa779b-c98c-4338-b216-12b15dc91f04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:42:24.850062 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1208/2427] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Aug 6 01:42:24 2022] DELETE /identity/v3/projects/4db9537bd65748f6b43061f3530f6891 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:42:24.853893 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1f5e2a-c671-4af7-a6c2-5b0f65430294 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:42:24.855498 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-ac1f5e2a-c671-4af7-a6c2-5b0f65430294 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:42:24.869988 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1f5e2a-c671-4af7-a6c2-5b0f65430294 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:42:24.956268 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620bad56-a472-4ee5-b676-b8f3558dc691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:42:24.957336 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620bad56-a472-4ee5-b676-b8f3558dc691 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:42:24.957773 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620bad56-a472-4ee5-b676-b8f3558dc691 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:42:24.958398 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620bad56-a472-4ee5-b676-b8f3558dc691 admin admin] PATH_INFO: `/v3/users/9c188b9cd883443ba28b1db11ccf555d` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:42:24.960080 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620bad56-a472-4ee5-b676-b8f3558dc691 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c188b9cd883443ba28b1db11ccf555d)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:42:24.962823 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620bad56-a472-4ee5-b676-b8f3558dc691 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:42:25.047967 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-620bad56-a472-4ee5-b676-b8f3558dc691 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c188b9cd883443ba28b1db11ccf555d'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:42:25.057324 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-620bad56-a472-4ee5-b676-b8f3558dc691 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c188b9cd883443ba28b1db11ccf555d'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:42:25.058342 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-620bad56-a472-4ee5-b676-b8f3558dc691 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c188b9cd883443ba28b1db11ccf555d'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:42:25.059133 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-620bad56-a472-4ee5-b676-b8f3558dc691 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c188b9cd883443ba28b1db11ccf555d'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:42:25.075439 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-620bad56-a472-4ee5-b676-b8f3558dc691 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c188b9cd883443ba28b1db11ccf555d'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:42:25.097353 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1220/2428] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Aug 6 01:42:24 2022] DELETE /identity/v3/users/9c188b9cd883443ba28b1db11ccf555d => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:42:25.314128 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faa779b-c98c-4338-b216-12b15dc91f04 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:42:25.315597 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-5faa779b-c98c-4338-b216-12b15dc91f04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:42:25.332379 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faa779b-c98c-4338-b216-12b15dc91f04 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:42:25.336637 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:42:25.424837 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121d671e-4a43-42c2-bd35-b2031dcca3c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:42:25.425532 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121d671e-4a43-42c2-bd35-b2031dcca3c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:42:25.425813 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121d671e-4a43-42c2-bd35-b2031dcca3c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:42:25.426092 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121d671e-4a43-42c2-bd35-b2031dcca3c5 admin admin] PATH_INFO: `/v3/projects/db7fea8cbb1b4c8b98849254635c1916` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:42:25.427603 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121d671e-4a43-42c2-bd35-b2031dcca3c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db7fea8cbb1b4c8b98849254635c1916)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:42:25.430371 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121d671e-4a43-42c2-bd35-b2031dcca3c5 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:42:25.498030 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-121d671e-4a43-42c2-bd35-b2031dcca3c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db7fea8cbb1b4c8b98849254635c1916', 'request_id': 'req-121d671e-4a43-42c2-bd35-b2031dcca3c5'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:42:25.508300 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-121d671e-4a43-42c2-bd35-b2031dcca3c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db7fea8cbb1b4c8b98849254635c1916', 'request_id': 'req-121d671e-4a43-42c2-bd35-b2031dcca3c5'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:42:25.515997 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-121d671e-4a43-42c2-bd35-b2031dcca3c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db7fea8cbb1b4c8b98849254635c1916', 'request_id': 'req-121d671e-4a43-42c2-bd35-b2031dcca3c5'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:42:25.537478 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-121d671e-4a43-42c2-bd35-b2031dcca3c5 admin admin] The token cache is being invalidate because project db7fea8cbb1b4c8b98849254635c1916 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:42:25.537781 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-121d671e-4a43-42c2-bd35-b2031dcca3c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:42:25.538441 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-121d671e-4a43-42c2-bd35-b2031dcca3c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:42:25.540481 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1209/2429] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Aug 6 01:42:25 2022] DELETE /identity/v3/projects/db7fea8cbb1b4c8b98849254635c1916 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:42:27.800574 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620bad56-a472-4ee5-b676-b8f3558dc691 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:42:27.801998 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-620bad56-a472-4ee5-b676-b8f3558dc691 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:42:27.860681 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:42:27.880627 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620bad56-a472-4ee5-b676-b8f3558dc691 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:42:27.949399 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b091f7c9-02f1-4eef-ac9c-5603da5db34d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa3b93ddd9464a2cba724d67b56b477a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:42:27.950124 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b091f7c9-02f1-4eef-ac9c-5603da5db34d service placement] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:42:27.950475 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b091f7c9-02f1-4eef-ac9c-5603da5db34d service placement] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:42:27.950817 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b091f7c9-02f1-4eef-ac9c-5603da5db34d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:42:27.953181 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-b091f7c9-02f1-4eef-ac9c-5603da5db34d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:42:27.967535 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b091f7c9-02f1-4eef-ac9c-5603da5db34d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:42:27.969130 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b091f7c9-02f1-4eef-ac9c-5603da5db34d service placement] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:42:27.980366 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1221/2430] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:42:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:42:28.313838 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb17514d-1e8e-4150-b792-c22d80f00d6c None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:42:28.314315 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb17514d-1e8e-4150-b792-c22d80f00d6c None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:42:28.314420 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb17514d-1e8e-4150-b792-c22d80f00d6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:42:28.334283 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-cb17514d-1e8e-4150-b792-c22d80f00d6c None None] MFA Rules not processed for user `8e86f38590824894a3b960e6774ff8a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:42:28.335476 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-cb17514d-1e8e-4150-b792-c22d80f00d6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:42:28.391820 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1210/2431] 10.222.0.26 () {58 vars in 1009 bytes} [Sat Aug 6 01:42:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:42:28.404175 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b091f7c9-02f1-4eef-ac9c-5603da5db34d service placement] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:42:28.405720 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-b091f7c9-02f1-4eef-ac9c-5603da5db34d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:42:28.478096 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b091f7c9-02f1-4eef-ac9c-5603da5db34d service placement] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:42:28.483178 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f33f23-1ae6-4e0a-9dda-e65e717d7a77 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b23f4478f49471fba6f43e6e4be99da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:42:28.483800 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f33f23-1ae6-4e0a-9dda-e65e717d7a77 service swift] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:42:28.484111 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f33f23-1ae6-4e0a-9dda-e65e717d7a77 service swift] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:42:28.484388 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f33f23-1ae6-4e0a-9dda-e65e717d7a77 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:42:28.493418 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f33f23-1ae6-4e0a-9dda-e65e717d7a77 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:42:28.495507 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f33f23-1ae6-4e0a-9dda-e65e717d7a77 service swift] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:42:28.504966 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1222/2432] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Aug 6 01:42:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 06 01:42:31.681957 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb17514d-1e8e-4150-b792-c22d80f00d6c None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:42:31.684330 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-cb17514d-1e8e-4150-b792-c22d80f00d6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:42:31.762557 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:42:31.786989 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb17514d-1e8e-4150-b792-c22d80f00d6c None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:42:31.792311 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840d089c-1c70-4a2b-8c21-aed74bad0534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:42:31.792969 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840d089c-1c70-4a2b-8c21-aed74bad0534 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:42:31.793273 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840d089c-1c70-4a2b-8c21-aed74bad0534 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:42:31.793549 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840d089c-1c70-4a2b-8c21-aed74bad0534 admin admin] PATH_INFO: `/v3/users/cbb2fa7d4e5b42388cb4a97ec3495d51` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:42:31.794983 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840d089c-1c70-4a2b-8c21-aed74bad0534 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbb2fa7d4e5b42388cb4a97ec3495d51)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:42:31.797141 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840d089c-1c70-4a2b-8c21-aed74bad0534 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:42:31.849659 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-840d089c-1c70-4a2b-8c21-aed74bad0534 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbb2fa7d4e5b42388cb4a97ec3495d51'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:42:31.862405 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-840d089c-1c70-4a2b-8c21-aed74bad0534 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbb2fa7d4e5b42388cb4a97ec3495d51'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:42:31.863259 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-840d089c-1c70-4a2b-8c21-aed74bad0534 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbb2fa7d4e5b42388cb4a97ec3495d51'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:42:31.878571 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-840d089c-1c70-4a2b-8c21-aed74bad0534 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbb2fa7d4e5b42388cb4a97ec3495d51'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:42:31.886989 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-840d089c-1c70-4a2b-8c21-aed74bad0534 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbb2fa7d4e5b42388cb4a97ec3495d51'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:42:31.905234 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1211/2433] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Aug 6 01:42:31 2022] DELETE /identity/v3/users/cbb2fa7d4e5b42388cb4a97ec3495d51 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:42:32.124203 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f33f23-1ae6-4e0a-9dda-e65e717d7a77 service swift] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:42:32.136591 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f33f23-1ae6-4e0a-9dda-e65e717d7a77 service swift] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:42:32.230930 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39c146d-7d77-43c8-b493-c732b28f58e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:42:32.231578 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39c146d-7d77-43c8-b493-c732b28f58e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:42:32.231860 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39c146d-7d77-43c8-b493-c732b28f58e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:42:32.232135 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39c146d-7d77-43c8-b493-c732b28f58e1 admin admin] PATH_INFO: `/v3/projects/722d0293c88d4711a7252d01bb91621d` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:42:32.233330 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39c146d-7d77-43c8-b493-c732b28f58e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=722d0293c88d4711a7252d01bb91621d)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:42:32.235229 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39c146d-7d77-43c8-b493-c732b28f58e1 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:42:32.260176 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:42:32.330013 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-f39c146d-7d77-43c8-b493-c732b28f58e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '722d0293c88d4711a7252d01bb91621d', 'request_id': 'req-f39c146d-7d77-43c8-b493-c732b28f58e1'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:42:32.361017 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-f39c146d-7d77-43c8-b493-c732b28f58e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '722d0293c88d4711a7252d01bb91621d', 'request_id': 'req-f39c146d-7d77-43c8-b493-c732b28f58e1'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:42:32.368991 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-f39c146d-7d77-43c8-b493-c732b28f58e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '722d0293c88d4711a7252d01bb91621d', 'request_id': 'req-f39c146d-7d77-43c8-b493-c732b28f58e1'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:42:32.411600 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-f39c146d-7d77-43c8-b493-c732b28f58e1 admin admin] The token cache is being invalidate because project 722d0293c88d4711a7252d01bb91621d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:42:32.411890 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-f39c146d-7d77-43c8-b493-c732b28f58e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:42:32.412577 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-f39c146d-7d77-43c8-b493-c732b28f58e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:42:32.417869 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1223/2434] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Aug 6 01:42:32 2022] DELETE /identity/v3/projects/722d0293c88d4711a7252d01bb91621d => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:42:32.420222 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840d089c-1c70-4a2b-8c21-aed74bad0534 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:42:32.422540 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-840d089c-1c70-4a2b-8c21-aed74bad0534 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:42:32.446707 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840d089c-1c70-4a2b-8c21-aed74bad0534 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:42:32.554900 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9657c8c5-3d72-446b-a1db-1b7486c35b0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:42:32.556390 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9657c8c5-3d72-446b-a1db-1b7486c35b0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:42:32.556939 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9657c8c5-3d72-446b-a1db-1b7486c35b0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:42:32.557412 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9657c8c5-3d72-446b-a1db-1b7486c35b0b admin admin] PATH_INFO: `/v3/users/ff7a95c0e54a40adb33ea6b152406554` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:42:32.558778 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9657c8c5-3d72-446b-a1db-1b7486c35b0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff7a95c0e54a40adb33ea6b152406554)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:42:32.560950 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9657c8c5-3d72-446b-a1db-1b7486c35b0b admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:42:32.616682 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-9657c8c5-3d72-446b-a1db-1b7486c35b0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff7a95c0e54a40adb33ea6b152406554'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:42:32.617402 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-9657c8c5-3d72-446b-a1db-1b7486c35b0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff7a95c0e54a40adb33ea6b152406554'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:42:32.618096 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-9657c8c5-3d72-446b-a1db-1b7486c35b0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff7a95c0e54a40adb33ea6b152406554'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:42:32.632586 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:42:32.634473 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-9657c8c5-3d72-446b-a1db-1b7486c35b0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff7a95c0e54a40adb33ea6b152406554'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:42:32.646412 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-9657c8c5-3d72-446b-a1db-1b7486c35b0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff7a95c0e54a40adb33ea6b152406554'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:42:32.663678 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1212/2435] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Aug 6 01:42:32 2022] DELETE /identity/v3/users/ff7a95c0e54a40adb33ea6b152406554 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:42:32.838731 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39c146d-7d77-43c8-b493-c732b28f58e1 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:42:32.840254 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-f39c146d-7d77-43c8-b493-c732b28f58e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:42:32.855104 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39c146d-7d77-43c8-b493-c732b28f58e1 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:42:32.920106 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53287b16-3693-42c1-b64a-b95b089d4770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:42:32.920773 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53287b16-3693-42c1-b64a-b95b089d4770 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:42:32.921071 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53287b16-3693-42c1-b64a-b95b089d4770 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:42:32.921471 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53287b16-3693-42c1-b64a-b95b089d4770 admin admin] PATH_INFO: `/v3/projects/9dfcddd9c01446688fe85ab203b609e8` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:42:32.923679 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53287b16-3693-42c1-b64a-b95b089d4770 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9dfcddd9c01446688fe85ab203b609e8)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:42:32.926005 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53287b16-3693-42c1-b64a-b95b089d4770 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:42:32.978511 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-53287b16-3693-42c1-b64a-b95b089d4770 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9dfcddd9c01446688fe85ab203b609e8', 'request_id': 'req-53287b16-3693-42c1-b64a-b95b089d4770'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:42:32.985094 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-53287b16-3693-42c1-b64a-b95b089d4770 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9dfcddd9c01446688fe85ab203b609e8', 'request_id': 'req-53287b16-3693-42c1-b64a-b95b089d4770'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:42:32.990820 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-53287b16-3693-42c1-b64a-b95b089d4770 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9dfcddd9c01446688fe85ab203b609e8', 'request_id': 'req-53287b16-3693-42c1-b64a-b95b089d4770'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:42:33.011293 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-53287b16-3693-42c1-b64a-b95b089d4770 admin admin] The token cache is being invalidate because project 9dfcddd9c01446688fe85ab203b609e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:42:33.011864 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-53287b16-3693-42c1-b64a-b95b089d4770 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:42:33.013188 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-53287b16-3693-42c1-b64a-b95b089d4770 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:42:33.016406 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1224/2436] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Aug 6 01:42:32 2022] DELETE /identity/v3/projects/9dfcddd9c01446688fe85ab203b609e8 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:42:42.660133 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=7,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:42:43.004535 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:DELETE=14,keystone:SELECT=66,keystone:INSERT=2 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:42:44.266527 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9657c8c5-3d72-446b-a1db-1b7486c35b0b admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:42:44.267951 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-9657c8c5-3d72-446b-a1db-1b7486c35b0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:42:44.323935 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9657c8c5-3d72-446b-a1db-1b7486c35b0b admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:42:44.371884 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53c74a7-7468-464f-898c-dc411051b872 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:42:44.372498 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53c74a7-7468-464f-898c-dc411051b872 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:42:44.372783 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53c74a7-7468-464f-898c-dc411051b872 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:42:44.373073 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53c74a7-7468-464f-898c-dc411051b872 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:42:44.374825 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-a53c74a7-7468-464f-898c-dc411051b872 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:42:44.444765 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53c74a7-7468-464f-898c-dc411051b872 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:42:44.446256 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53c74a7-7468-464f-898c-dc411051b872 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:42:44.457114 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1213/2437] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:42:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:42:45.642035 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53287b16-3693-42c1-b64a-b95b089d4770 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:42:45.643526 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-53287b16-3693-42c1-b64a-b95b089d4770 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:42:45.656188 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53287b16-3693-42c1-b64a-b95b089d4770 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:42:45.659834 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e1b4f9-3265-48be-9bdc-b2bcce2b666f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:42:45.660436 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e1b4f9-3265-48be-9bdc-b2bcce2b666f service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:42:45.660715 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e1b4f9-3265-48be-9bdc-b2bcce2b666f service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:42:45.660986 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e1b4f9-3265-48be-9bdc-b2bcce2b666f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:42:45.662721 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-f9e1b4f9-3265-48be-9bdc-b2bcce2b666f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:42:45.679472 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e1b4f9-3265-48be-9bdc-b2bcce2b666f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:42:45.681945 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e1b4f9-3265-48be-9bdc-b2bcce2b666f service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:42:45.692570 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1225/2438] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:42:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:42:49.026978 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e359833a-ddb6-4725-acc2-749a7b02f480 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:42:49.027444 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e359833a-ddb6-4725-acc2-749a7b02f480 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:42:49.027546 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e359833a-ddb6-4725-acc2-749a7b02f480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:42:49.046846 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-e359833a-ddb6-4725-acc2-749a7b02f480 None None] MFA Rules not processed for user `8e86f38590824894a3b960e6774ff8a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:42:49.048151 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-e359833a-ddb6-4725-acc2-749a7b02f480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:42:49.102832 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1214/2439] 10.222.0.26 () {58 vars in 967 bytes} [Sat Aug 6 01:42:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:42:49.115025 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e1b4f9-3265-48be-9bdc-b2bcce2b666f service neutron] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:42:49.116424 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-f9e1b4f9-3265-48be-9bdc-b2bcce2b666f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:42:49.186645 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e1b4f9-3265-48be-9bdc-b2bcce2b666f service neutron] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:42:49.191872 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec7a860-30c8-4447-96c7-31d3609f556a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b23f4478f49471fba6f43e6e4be99da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:42:49.192502 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec7a860-30c8-4447-96c7-31d3609f556a service swift] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:42:49.192785 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec7a860-30c8-4447-96c7-31d3609f556a service swift] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:42:49.193077 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec7a860-30c8-4447-96c7-31d3609f556a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:42:49.202311 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec7a860-30c8-4447-96c7-31d3609f556a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:42:49.203898 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec7a860-30c8-4447-96c7-31d3609f556a service swift] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:42:49.212166 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1226/2440] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Aug 6 01:42:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 06 01:42:59.096089 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:42:59.208721 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=38 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:43:23.331183 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889e9760-6090-48e5-abd1-f44b80b8d90a None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:43:23.331726 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889e9760-6090-48e5-abd1-f44b80b8d90a None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:43:23.331822 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889e9760-6090-48e5-abd1-f44b80b8d90a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:43:23.352014 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-889e9760-6090-48e5-abd1-f44b80b8d90a None None] MFA Rules not processed for user `d2235317fe8d4d7ebae7bb5b80dfd7e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:43:23.353261 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-889e9760-6090-48e5-abd1-f44b80b8d90a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:43:23.413963 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1215/2441] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:43:23 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:43:23.965443 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec7a860-30c8-4447-96c7-31d3609f556a service swift] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:43:23.975620 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec7a860-30c8-4447-96c7-31d3609f556a service swift] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:43:23.979208 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca53fda-4624-468a-aafe-4814e83ce5ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:43:23.979816 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca53fda-4624-468a-aafe-4814e83ce5ff service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:43:23.980152 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca53fda-4624-468a-aafe-4814e83ce5ff service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:43:23.980551 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca53fda-4624-468a-aafe-4814e83ce5ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:43:23.982474 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-9ca53fda-4624-468a-aafe-4814e83ce5ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:43:23.991924 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca53fda-4624-468a-aafe-4814e83ce5ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:43:23.993592 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca53fda-4624-468a-aafe-4814e83ce5ff service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:43:24.002441 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1227/2442] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:43:23 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:43:24.327823 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889e9760-6090-48e5-abd1-f44b80b8d90a None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:43:24.329258 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-889e9760-6090-48e5-abd1-f44b80b8d90a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:43:24.420400 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889e9760-6090-48e5-abd1-f44b80b8d90a None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:43:24.425417 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749de77d-98be-4280-81ed-88dfb1a0e441 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa3b93ddd9464a2cba724d67b56b477a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:43:24.426073 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749de77d-98be-4280-81ed-88dfb1a0e441 service placement] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:43:24.426357 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749de77d-98be-4280-81ed-88dfb1a0e441 service placement] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:43:24.426633 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749de77d-98be-4280-81ed-88dfb1a0e441 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:43:24.428400 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-749de77d-98be-4280-81ed-88dfb1a0e441 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:43:24.440568 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749de77d-98be-4280-81ed-88dfb1a0e441 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:43:24.442167 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749de77d-98be-4280-81ed-88dfb1a0e441 service placement] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:43:24.452269 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1216/2443] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:43:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:43:28.541943 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca53fda-4624-468a-aafe-4814e83ce5ff service neutron] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:43:28.553928 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca53fda-4624-468a-aafe-4814e83ce5ff service neutron] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:43:28.559613 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7e1a6b-aec1-4b07-8612-d43566e8defb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:43:28.560248 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7e1a6b-aec1-4b07-8612-d43566e8defb service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:43:28.560536 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7e1a6b-aec1-4b07-8612-d43566e8defb service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:43:28.560960 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7e1a6b-aec1-4b07-8612-d43566e8defb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:43:28.563114 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-ca7e1a6b-aec1-4b07-8612-d43566e8defb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:43:28.575540 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7e1a6b-aec1-4b07-8612-d43566e8defb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:43:28.577119 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7e1a6b-aec1-4b07-8612-d43566e8defb service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:43:28.587300 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1228/2444] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:43:28 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:43:33.855403 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749de77d-98be-4280-81ed-88dfb1a0e441 service placement] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:43:33.864746 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749de77d-98be-4280-81ed-88dfb1a0e441 service placement] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:43:33.868305 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679a727d-7482-4120-836c-bb727b8f62d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:43:33.868912 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679a727d-7482-4120-836c-bb727b8f62d7 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:43:33.869231 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679a727d-7482-4120-836c-bb727b8f62d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:43:33.869508 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679a727d-7482-4120-836c-bb727b8f62d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:43:33.871624 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-679a727d-7482-4120-836c-bb727b8f62d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:43:33.934214 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679a727d-7482-4120-836c-bb727b8f62d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:43:33.935639 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679a727d-7482-4120-836c-bb727b8f62d7 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:43:33.945166 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1217/2445] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:43:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:43:37.000704 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7e1a6b-aec1-4b07-8612-d43566e8defb service neutron] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:43:37.002157 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-ca7e1a6b-aec1-4b07-8612-d43566e8defb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:43:37.018044 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7e1a6b-aec1-4b07-8612-d43566e8defb service neutron] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:43:37.022555 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7972600-2900-461b-9413-ab710c52abc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:43:37.023194 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7972600-2900-461b-9413-ab710c52abc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:43:37.023478 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7972600-2900-461b-9413-ab710c52abc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:43:37.023747 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7972600-2900-461b-9413-ab710c52abc6 admin admin] PATH_INFO: `/v3/users/2455ea8abba547669fe22d2065a726c6` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:43:37.038543 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7972600-2900-461b-9413-ab710c52abc6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2455ea8abba547669fe22d2065a726c6)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:43:37.040479 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7972600-2900-461b-9413-ab710c52abc6 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:43:37.084621 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-e7972600-2900-461b-9413-ab710c52abc6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2455ea8abba547669fe22d2065a726c6'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:43:37.091957 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-e7972600-2900-461b-9413-ab710c52abc6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2455ea8abba547669fe22d2065a726c6'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:43:37.092623 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-e7972600-2900-461b-9413-ab710c52abc6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2455ea8abba547669fe22d2065a726c6'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:43:37.093274 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-e7972600-2900-461b-9413-ab710c52abc6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2455ea8abba547669fe22d2065a726c6'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:43:37.105815 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-e7972600-2900-461b-9413-ab710c52abc6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2455ea8abba547669fe22d2065a726c6'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:43:37.118149 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1229/2446] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Aug 6 01:43:36 2022] DELETE /identity/v3/users/2455ea8abba547669fe22d2065a726c6 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:43:37.296906 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679a727d-7482-4120-836c-bb727b8f62d7 service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:43:37.308021 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679a727d-7482-4120-836c-bb727b8f62d7 service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:43:37.336642 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:43:37.371109 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348a309c-322c-487d-a3cc-09fd7f8a8954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:43:37.371757 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348a309c-322c-487d-a3cc-09fd7f8a8954 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:43:37.372039 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348a309c-322c-487d-a3cc-09fd7f8a8954 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:43:37.372314 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348a309c-322c-487d-a3cc-09fd7f8a8954 admin admin] PATH_INFO: `/v3/projects/4bd83104ff2241519bcdd4e232eb6895` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:43:37.383340 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348a309c-322c-487d-a3cc-09fd7f8a8954 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4bd83104ff2241519bcdd4e232eb6895)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:43:37.385325 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348a309c-322c-487d-a3cc-09fd7f8a8954 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:43:37.435065 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-348a309c-322c-487d-a3cc-09fd7f8a8954 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4bd83104ff2241519bcdd4e232eb6895', 'request_id': 'req-348a309c-322c-487d-a3cc-09fd7f8a8954'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:43:37.443770 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-348a309c-322c-487d-a3cc-09fd7f8a8954 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4bd83104ff2241519bcdd4e232eb6895', 'request_id': 'req-348a309c-322c-487d-a3cc-09fd7f8a8954'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:43:37.449731 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-348a309c-322c-487d-a3cc-09fd7f8a8954 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4bd83104ff2241519bcdd4e232eb6895', 'request_id': 'req-348a309c-322c-487d-a3cc-09fd7f8a8954'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:43:37.462161 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-348a309c-322c-487d-a3cc-09fd7f8a8954 admin admin] The token cache is being invalidate because project 4bd83104ff2241519bcdd4e232eb6895 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:43:37.462507 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-348a309c-322c-487d-a3cc-09fd7f8a8954 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:43:37.463137 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-348a309c-322c-487d-a3cc-09fd7f8a8954 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:43:37.465508 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1218/2447] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Aug 6 01:43:37 2022] DELETE /identity/v3/projects/4bd83104ff2241519bcdd4e232eb6895 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:43:37.470583 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc840ecb-77c9-4c41-bdf3-abb9f102be2e None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:43:37.470875 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc840ecb-77c9-4c41-bdf3-abb9f102be2e None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:43:37.471157 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc840ecb-77c9-4c41-bdf3-abb9f102be2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:43:37.491421 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-fc840ecb-77c9-4c41-bdf3-abb9f102be2e None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:43:37.492636 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-fc840ecb-77c9-4c41-bdf3-abb9f102be2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:43:37.585697 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1230/2448] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:43:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:43:37.594305 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a85d767-7c48-4947-8645-444a8cf0dde8 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:43:37.594595 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a85d767-7c48-4947-8645-444a8cf0dde8 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:43:37.594869 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a85d767-7c48-4947-8645-444a8cf0dde8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:43:37.615354 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-6a85d767-7c48-4947-8645-444a8cf0dde8 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:43:37.616586 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-6a85d767-7c48-4947-8645-444a8cf0dde8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:43:37.627810 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1219/2449] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:43:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:43:37.631795 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc840ecb-77c9-4c41-bdf3-abb9f102be2e None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:43:37.643359 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc840ecb-77c9-4c41-bdf3-abb9f102be2e None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:43:37.654256 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f802c079-4ba0-49a5-9bfe-6c8f5021b6a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:43:37.654962 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f802c079-4ba0-49a5-9bfe-6c8f5021b6a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:43:37.655310 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f802c079-4ba0-49a5-9bfe-6c8f5021b6a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:43:37.655858 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f802c079-4ba0-49a5-9bfe-6c8f5021b6a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:43:37.656630 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f802c079-4ba0-49a5-9bfe-6c8f5021b6a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:43:37.656999 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f802c079-4ba0-49a5-9bfe-6c8f5021b6a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:43:37.658631 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f802c079-4ba0-49a5-9bfe-6c8f5021b6a2 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:43:37.674837 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1231/2450] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Aug 6 01:43:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:43:37.675769 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a85d767-7c48-4947-8645-444a8cf0dde8 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:43:37.685870 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a85d767-7c48-4947-8645-444a8cf0dde8 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:43:37.691088 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc49c58-84c3-4354-87c6-438abe15955d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:43:37.691798 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc49c58-84c3-4354-87c6-438abe15955d admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:43:37.692076 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc49c58-84c3-4354-87c6-438abe15955d admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:43:37.692342 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc49c58-84c3-4354-87c6-438abe15955d admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:43:37.693008 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc49c58-84c3-4354-87c6-438abe15955d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:43:37.694934 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc49c58-84c3-4354-87c6-438abe15955d admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:43:37.734620 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1220/2451] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Aug 6 01:43:37 2022] POST /identity/v3/projects => generated 405 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:43:37.738946 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f802c079-4ba0-49a5-9bfe-6c8f5021b6a2 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:43:37.750618 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f802c079-4ba0-49a5-9bfe-6c8f5021b6a2 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:43:37.769333 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:43:37.811509 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce27c298-35d4-4582-a708-d3a82040c2fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:43:37.812233 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce27c298-35d4-4582-a708-d3a82040c2fd admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:43:37.812521 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce27c298-35d4-4582-a708-d3a82040c2fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:43:37.812794 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce27c298-35d4-4582-a708-d3a82040c2fd admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:43:37.813635 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce27c298-35d4-4582-a708-d3a82040c2fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:43:37.815615 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce27c298-35d4-4582-a708-d3a82040c2fd admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:43:37.841900 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1232/2452] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:43:37 2022] POST /identity/v3/users => generated 303 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:43:37.843117 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc49c58-84c3-4354-87c6-438abe15955d admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:43:37.853600 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc49c58-84c3-4354-87c6-438abe15955d admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:43:37.859051 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5593d0-21e8-4642-82a9-ebfe94591f02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:43:37.859673 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5593d0-21e8-4642-82a9-ebfe94591f02 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:43:37.859958 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5593d0-21e8-4642-82a9-ebfe94591f02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:43:37.860233 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5593d0-21e8-4642-82a9-ebfe94591f02 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:43:37.860944 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5593d0-21e8-4642-82a9-ebfe94591f02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:43:37.862603 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5593d0-21e8-4642-82a9-ebfe94591f02 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:43:37.874138 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1221/2453] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:43:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:43:37.878190 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce27c298-35d4-4582-a708-d3a82040c2fd admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:43:37.889374 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce27c298-35d4-4582-a708-d3a82040c2fd admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:43:37.894748 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23192682-b4bf-4716-96fd-b24d7b4e05f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:43:37.895778 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23192682-b4bf-4716-96fd-b24d7b4e05f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:43:37.896320 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23192682-b4bf-4716-96fd-b24d7b4e05f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:43:37.896602 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23192682-b4bf-4716-96fd-b24d7b4e05f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:43:37.897332 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23192682-b4bf-4716-96fd-b24d7b4e05f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:43:37.898940 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23192682-b4bf-4716-96fd-b24d7b4e05f3 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:43:37.914092 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1233/2454] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:43:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:43:37.915327 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5593d0-21e8-4642-82a9-ebfe94591f02 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:43:37.925981 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5593d0-21e8-4642-82a9-ebfe94591f02 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:43:37.930867 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339c82dd-1867-4716-a28f-9ba47170bad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:43:37.931537 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339c82dd-1867-4716-a28f-9ba47170bad8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:43:37.931825 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339c82dd-1867-4716-a28f-9ba47170bad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:43:37.932108 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339c82dd-1867-4716-a28f-9ba47170bad8 admin admin] PATH_INFO: `/v3/projects/88267c1b4f0b4f2a95a2c3bdee0eca4f/users/d9718a61f86045b1ade6cb74e44f789c/roles/61b3b93d37a047e4bb8369b46fe20db9` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:43:37.947589 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339c82dd-1867-4716-a28f-9ba47170bad8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88267c1b4f0b4f2a95a2c3bdee0eca4f, user_id=d9718a61f86045b1ade6cb74e44f789c, role_id=61b3b93d37a047e4bb8369b46fe20db9)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:43:37.950869 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339c82dd-1867-4716-a28f-9ba47170bad8 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:43:37.963831 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1222/2455] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:43:37 2022] PUT /identity/v3/projects/88267c1b4f0b4f2a95a2c3bdee0eca4f/users/d9718a61f86045b1ade6cb74e44f789c/roles/61b3b93d37a047e4bb8369b46fe20db9 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:43:37.969331 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33a586d-8741-4bfd-a2f0-13d1b48feac2 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:43:37.969621 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33a586d-8741-4bfd-a2f0-13d1b48feac2 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:43:37.969897 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33a586d-8741-4bfd-a2f0-13d1b48feac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:43:37.991353 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-f33a586d-8741-4bfd-a2f0-13d1b48feac2 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:43:37.992465 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-f33a586d-8741-4bfd-a2f0-13d1b48feac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:43:38.054616 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1234/2456] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:43:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:43:38.066589 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339c82dd-1867-4716-a28f-9ba47170bad8 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:43:38.068826 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-339c82dd-1867-4716-a28f-9ba47170bad8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:43:38.111671 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:43:38.130653 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339c82dd-1867-4716-a28f-9ba47170bad8 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:43:38.134376 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83bf7f0-c839-4816-a978-131db26ab086 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:43:38.135001 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83bf7f0-c839-4816-a978-131db26ab086 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:43:38.135285 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83bf7f0-c839-4816-a978-131db26ab086 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:43:38.135564 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83bf7f0-c839-4816-a978-131db26ab086 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:43:38.145893 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83bf7f0-c839-4816-a978-131db26ab086 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:43:38.147376 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83bf7f0-c839-4816-a978-131db26ab086 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:43:38.157401 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1223/2457] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:43:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:43:38.822117 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbf09f3-55fa-4be8-a89d-906b4a5852c9 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:43:38.822675 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbf09f3-55fa-4be8-a89d-906b4a5852c9 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:43:38.823111 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbf09f3-55fa-4be8-a89d-906b4a5852c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:43:38.871065 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-afbf09f3-55fa-4be8-a89d-906b4a5852c9 None None] MFA Rules not processed for user `d9718a61f86045b1ade6cb74e44f789c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:43:38.873276 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-afbf09f3-55fa-4be8-a89d-906b4a5852c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:43:38.968567 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1235/2458] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:43:38 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:43:38.973804 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83bf7f0-c839-4816-a978-131db26ab086 service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:43:38.988236 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83bf7f0-c839-4816-a978-131db26ab086 service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:43:38.993996 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db2c4dc-a2be-4935-901f-898064c9126b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:43:38.994832 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db2c4dc-a2be-4935-901f-898064c9126b admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:43:38.995445 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db2c4dc-a2be-4935-901f-898064c9126b admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:43:38.996880 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db2c4dc-a2be-4935-901f-898064c9126b admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:43:38.997955 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db2c4dc-a2be-4935-901f-898064c9126b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:43:39.000518 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db2c4dc-a2be-4935-901f-898064c9126b admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:43:39.038385 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1224/2459] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Aug 6 01:43:38 2022] POST /identity/v3/projects => generated 401 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:43:39.039531 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbf09f3-55fa-4be8-a89d-906b4a5852c9 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:43:39.053149 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbf09f3-55fa-4be8-a89d-906b4a5852c9 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:43:39.088496 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:43:39.145232 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361fd26a-2e61-4ae3-865e-aaeeb9aabf34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:43:39.145902 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361fd26a-2e61-4ae3-865e-aaeeb9aabf34 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:43:39.146183 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361fd26a-2e61-4ae3-865e-aaeeb9aabf34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:43:39.146464 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361fd26a-2e61-4ae3-865e-aaeeb9aabf34 admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:43:39.147349 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361fd26a-2e61-4ae3-865e-aaeeb9aabf34 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:43:39.149565 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361fd26a-2e61-4ae3-865e-aaeeb9aabf34 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:43:39.174351 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1236/2460] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:43:39 2022] POST /identity/v3/users => generated 307 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:43:39.178274 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db2c4dc-a2be-4935-901f-898064c9126b admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:43:39.189744 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db2c4dc-a2be-4935-901f-898064c9126b admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:43:39.195105 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f70ec4-4c2f-4e43-82fd-46e1fbd38b24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:43:39.195734 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f70ec4-4c2f-4e43-82fd-46e1fbd38b24 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:43:39.196018 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f70ec4-4c2f-4e43-82fd-46e1fbd38b24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:43:39.196299 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f70ec4-4c2f-4e43-82fd-46e1fbd38b24 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:43:39.196994 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f70ec4-4c2f-4e43-82fd-46e1fbd38b24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:43:39.199179 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f70ec4-4c2f-4e43-82fd-46e1fbd38b24 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:43:39.211388 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1225/2461] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:43:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:43:39.216171 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361fd26a-2e61-4ae3-865e-aaeeb9aabf34 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:43:39.226452 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361fd26a-2e61-4ae3-865e-aaeeb9aabf34 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:43:39.231873 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4432d7-2d83-488d-b964-f5c6d81f95aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:43:39.232599 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4432d7-2d83-488d-b964-f5c6d81f95aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:43:39.232884 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4432d7-2d83-488d-b964-f5c6d81f95aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:43:39.233188 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4432d7-2d83-488d-b964-f5c6d81f95aa admin admin] PATH_INFO: `/v3/projects/848dda017557475d934f68d21a3472f6/users/41c446bd06d640939a007d0209e47598/roles/4559b9a0837249f895a13c70096f8d38` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:43:39.250472 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4432d7-2d83-488d-b964-f5c6d81f95aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=848dda017557475d934f68d21a3472f6, user_id=41c446bd06d640939a007d0209e47598, role_id=4559b9a0837249f895a13c70096f8d38)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:43:39.253520 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4432d7-2d83-488d-b964-f5c6d81f95aa admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:43:39.268173 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1237/2462] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:43:39 2022] PUT /identity/v3/projects/848dda017557475d934f68d21a3472f6/users/41c446bd06d640939a007d0209e47598/roles/4559b9a0837249f895a13c70096f8d38 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:43:39.272118 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f70ec4-4c2f-4e43-82fd-46e1fbd38b24 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:43:39.282903 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f70ec4-4c2f-4e43-82fd-46e1fbd38b24 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:43:39.345599 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0c8ff3-7bd0-4558-8909-099b24a19232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:43:39.346323 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0c8ff3-7bd0-4558-8909-099b24a19232 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:43:39.346622 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0c8ff3-7bd0-4558-8909-099b24a19232 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:43:39.346899 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0c8ff3-7bd0-4558-8909-099b24a19232 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:43:39.347676 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0c8ff3-7bd0-4558-8909-099b24a19232 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:43:39.349337 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0c8ff3-7bd0-4558-8909-099b24a19232 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:43:39.360650 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1226/2463] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:43:39 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 01:43:39.364906 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4432d7-2d83-488d-b964-f5c6d81f95aa admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:43:39.376599 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4432d7-2d83-488d-b964-f5c6d81f95aa admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:43:39.381496 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a880649-57ad-484c-b9f4-a27ea650c6b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:43:39.382177 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a880649-57ad-484c-b9f4-a27ea650c6b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:43:39.382483 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a880649-57ad-484c-b9f4-a27ea650c6b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:43:39.382783 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a880649-57ad-484c-b9f4-a27ea650c6b4 admin admin] PATH_INFO: `/v3/projects/848dda017557475d934f68d21a3472f6/users/41c446bd06d640939a007d0209e47598/roles/4559b9a0837249f895a13c70096f8d38` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:43:39.384737 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a880649-57ad-484c-b9f4-a27ea650c6b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=848dda017557475d934f68d21a3472f6, user_id=41c446bd06d640939a007d0209e47598, role_id=4559b9a0837249f895a13c70096f8d38)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:43:39.387699 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a880649-57ad-484c-b9f4-a27ea650c6b4 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:43:39.398946 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1238/2464] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:43:39 2022] PUT /identity/v3/projects/848dda017557475d934f68d21a3472f6/users/41c446bd06d640939a007d0209e47598/roles/4559b9a0837249f895a13c70096f8d38 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:43:40.062886 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383599ca-5bd0-45dc-b8cd-51ad292df9eb None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:43:40.063481 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383599ca-5bd0-45dc-b8cd-51ad292df9eb None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:43:40.063481 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383599ca-5bd0-45dc-b8cd-51ad292df9eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:43:40.092479 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-383599ca-5bd0-45dc-b8cd-51ad292df9eb None None] MFA Rules not processed for user `41c446bd06d640939a007d0209e47598`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:43:40.093878 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-383599ca-5bd0-45dc-b8cd-51ad292df9eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:43:40.165247 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1227/2465] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:43:40 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:43:41.190569 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a880649-57ad-484c-b9f4-a27ea650c6b4 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:43:41.257582 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a880649-57ad-484c-b9f4-a27ea650c6b4 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:43:41.262763 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c8002a-402d-4807-a52b-73c1495c0ab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:43:41.263399 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c8002a-402d-4807-a52b-73c1495c0ab1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:43:41.263745 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c8002a-402d-4807-a52b-73c1495c0ab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:43:41.264029 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c8002a-402d-4807-a52b-73c1495c0ab1 admin admin] PATH_INFO: `/v3/users/d9718a61f86045b1ade6cb74e44f789c` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:43:41.265240 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c8002a-402d-4807-a52b-73c1495c0ab1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9718a61f86045b1ade6cb74e44f789c)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:43:41.267151 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c8002a-402d-4807-a52b-73c1495c0ab1 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:43:41.312328 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-58c8002a-402d-4807-a52b-73c1495c0ab1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9718a61f86045b1ade6cb74e44f789c'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:43:41.318995 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-58c8002a-402d-4807-a52b-73c1495c0ab1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9718a61f86045b1ade6cb74e44f789c'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:43:41.319644 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-58c8002a-402d-4807-a52b-73c1495c0ab1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9718a61f86045b1ade6cb74e44f789c'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:43:41.320301 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-58c8002a-402d-4807-a52b-73c1495c0ab1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9718a61f86045b1ade6cb74e44f789c'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:43:41.333104 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-58c8002a-402d-4807-a52b-73c1495c0ab1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9718a61f86045b1ade6cb74e44f789c'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:43:41.345846 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1239/2466] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Aug 6 01:43:41 2022] DELETE /identity/v3/users/d9718a61f86045b1ade6cb74e44f789c => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:43:41.521565 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383599ca-5bd0-45dc-b8cd-51ad292df9eb None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:43:41.533564 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383599ca-5bd0-45dc-b8cd-51ad292df9eb None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:43:41.561389 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:43:41.592743 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4205849-40fc-4875-b572-a631c55f8579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:43:41.593549 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4205849-40fc-4875-b572-a631c55f8579 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:43:41.593837 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4205849-40fc-4875-b572-a631c55f8579 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:43:41.594118 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4205849-40fc-4875-b572-a631c55f8579 admin admin] PATH_INFO: `/v3/projects/88267c1b4f0b4f2a95a2c3bdee0eca4f` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:43:41.595291 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4205849-40fc-4875-b572-a631c55f8579 admin admin] RBAC: Authorizing `identity:delete_project(project_id=88267c1b4f0b4f2a95a2c3bdee0eca4f)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:43:41.597246 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4205849-40fc-4875-b572-a631c55f8579 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:43:41.643061 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-b4205849-40fc-4875-b572-a631c55f8579 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88267c1b4f0b4f2a95a2c3bdee0eca4f', 'request_id': 'req-b4205849-40fc-4875-b572-a631c55f8579'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:43:41.649254 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-b4205849-40fc-4875-b572-a631c55f8579 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88267c1b4f0b4f2a95a2c3bdee0eca4f', 'request_id': 'req-b4205849-40fc-4875-b572-a631c55f8579'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:43:41.655022 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-b4205849-40fc-4875-b572-a631c55f8579 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88267c1b4f0b4f2a95a2c3bdee0eca4f', 'request_id': 'req-b4205849-40fc-4875-b572-a631c55f8579'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:43:41.667585 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-b4205849-40fc-4875-b572-a631c55f8579 admin admin] The token cache is being invalidate because project 88267c1b4f0b4f2a95a2c3bdee0eca4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:43:41.668154 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-b4205849-40fc-4875-b572-a631c55f8579 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:43:41.668734 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-b4205849-40fc-4875-b572-a631c55f8579 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:43:41.670272 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1228/2467] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Aug 6 01:43:41 2022] DELETE /identity/v3/projects/88267c1b4f0b4f2a95a2c3bdee0eca4f => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:43:41.674313 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c8002a-402d-4807-a52b-73c1495c0ab1 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:43:41.675720 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-58c8002a-402d-4807-a52b-73c1495c0ab1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:43:41.684911 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=84,keystone:INSERT=6,keystone:DELETE=10 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:43:41.690870 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c8002a-402d-4807-a52b-73c1495c0ab1 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:43:41.748134 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdcc59d-46c4-4ec7-89ff-9bf8d0dc5a65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:43:41.748779 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdcc59d-46c4-4ec7-89ff-9bf8d0dc5a65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:43:41.749097 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdcc59d-46c4-4ec7-89ff-9bf8d0dc5a65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:43:41.749380 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdcc59d-46c4-4ec7-89ff-9bf8d0dc5a65 admin admin] PATH_INFO: `/v3/users/41c446bd06d640939a007d0209e47598` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:43:41.750558 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdcc59d-46c4-4ec7-89ff-9bf8d0dc5a65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41c446bd06d640939a007d0209e47598)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:43:41.752492 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdcc59d-46c4-4ec7-89ff-9bf8d0dc5a65 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:43:41.798568 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-9fdcc59d-46c4-4ec7-89ff-9bf8d0dc5a65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41c446bd06d640939a007d0209e47598'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:43:41.806121 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-9fdcc59d-46c4-4ec7-89ff-9bf8d0dc5a65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41c446bd06d640939a007d0209e47598'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:43:41.807004 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-9fdcc59d-46c4-4ec7-89ff-9bf8d0dc5a65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41c446bd06d640939a007d0209e47598'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:43:41.807689 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-9fdcc59d-46c4-4ec7-89ff-9bf8d0dc5a65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41c446bd06d640939a007d0209e47598'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:43:41.819969 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-9fdcc59d-46c4-4ec7-89ff-9bf8d0dc5a65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41c446bd06d640939a007d0209e47598'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:43:41.832863 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1240/2468] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Aug 6 01:43:41 2022] DELETE /identity/v3/users/41c446bd06d640939a007d0209e47598 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:43:42.005574 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4205849-40fc-4875-b572-a631c55f8579 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:43:42.007008 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-b4205849-40fc-4875-b572-a631c55f8579 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:43:42.020354 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4205849-40fc-4875-b572-a631c55f8579 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:43:42.077034 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04b6407-7692-4e32-bd6c-1050c8ec2377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:43:42.077695 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04b6407-7692-4e32-bd6c-1050c8ec2377 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:43:42.077979 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04b6407-7692-4e32-bd6c-1050c8ec2377 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:43:42.078256 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04b6407-7692-4e32-bd6c-1050c8ec2377 admin admin] PATH_INFO: `/v3/projects/848dda017557475d934f68d21a3472f6` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:43:42.079400 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04b6407-7692-4e32-bd6c-1050c8ec2377 admin admin] RBAC: Authorizing `identity:delete_project(project_id=848dda017557475d934f68d21a3472f6)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:43:42.081337 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04b6407-7692-4e32-bd6c-1050c8ec2377 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:43:42.129663 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-d04b6407-7692-4e32-bd6c-1050c8ec2377 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '848dda017557475d934f68d21a3472f6', 'request_id': 'req-d04b6407-7692-4e32-bd6c-1050c8ec2377'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:43:42.135605 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-d04b6407-7692-4e32-bd6c-1050c8ec2377 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '848dda017557475d934f68d21a3472f6', 'request_id': 'req-d04b6407-7692-4e32-bd6c-1050c8ec2377'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:43:42.140960 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:43:42.142683 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-d04b6407-7692-4e32-bd6c-1050c8ec2377 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '848dda017557475d934f68d21a3472f6', 'request_id': 'req-d04b6407-7692-4e32-bd6c-1050c8ec2377'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:43:42.156884 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-d04b6407-7692-4e32-bd6c-1050c8ec2377 admin admin] The token cache is being invalidate because project 848dda017557475d934f68d21a3472f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:43:42.157274 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-d04b6407-7692-4e32-bd6c-1050c8ec2377 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:43:42.157904 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-d04b6407-7692-4e32-bd6c-1050c8ec2377 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:43:42.159411 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1229/2469] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Aug 6 01:43:42 2022] DELETE /identity/v3/projects/848dda017557475d934f68d21a3472f6 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:43:42.164252 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d71246-ebc8-42d3-b16d-52f3b8dcfafb None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:43:42.164547 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d71246-ebc8-42d3-b16d-52f3b8dcfafb None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:43:42.164829 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d71246-ebc8-42d3-b16d-52f3b8dcfafb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:43:42.184440 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-c4d71246-ebc8-42d3-b16d-52f3b8dcfafb None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:43:42.185705 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-c4d71246-ebc8-42d3-b16d-52f3b8dcfafb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:43:42.246905 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1241/2470] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:43:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:43:42.254535 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a6f0f1-f1b1-4cc6-bce5-31801429adcf None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:43:42.254825 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a6f0f1-f1b1-4cc6-bce5-31801429adcf None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:43:42.255101 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a6f0f1-f1b1-4cc6-bce5-31801429adcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:43:42.275099 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-23a6f0f1-f1b1-4cc6-bce5-31801429adcf None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:43:42.276314 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-23a6f0f1-f1b1-4cc6-bce5-31801429adcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:43:42.284825 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1230/2471] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:43:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:43:42.288992 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d71246-ebc8-42d3-b16d-52f3b8dcfafb None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:43:42.298601 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d71246-ebc8-42d3-b16d-52f3b8dcfafb None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:43:42.303180 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc831fd1-4233-4892-a198-1905dfe112ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:43:42.303789 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc831fd1-4233-4892-a198-1905dfe112ce admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:43:42.304073 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc831fd1-4233-4892-a198-1905dfe112ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:43:42.304350 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc831fd1-4233-4892-a198-1905dfe112ce admin admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:43:42.305067 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc831fd1-4233-4892-a198-1905dfe112ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:43:42.305377 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc831fd1-4233-4892-a198-1905dfe112ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:43:42.306974 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc831fd1-4233-4892-a198-1905dfe112ce admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:43:42.318527 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1242/2472] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Aug 6 01:43:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:43:42.322743 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a6f0f1-f1b1-4cc6-bce5-31801429adcf None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:43:42.332333 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a6f0f1-f1b1-4cc6-bce5-31801429adcf None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:43:42.336933 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b61f946-797b-462a-b14a-c635c1bf18da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:43:42.337566 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b61f946-797b-462a-b14a-c635c1bf18da admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:43:42.337846 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b61f946-797b-462a-b14a-c635c1bf18da admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:43:42.338121 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b61f946-797b-462a-b14a-c635c1bf18da admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:43:42.338780 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b61f946-797b-462a-b14a-c635c1bf18da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:43:42.341410 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b61f946-797b-462a-b14a-c635c1bf18da admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:43:42.367684 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1231/2473] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Aug 6 01:43:42 2022] POST /identity/v3/projects => generated 379 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:43:42.372149 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc831fd1-4233-4892-a198-1905dfe112ce admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:43:42.381914 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc831fd1-4233-4892-a198-1905dfe112ce admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:43:42.387518 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:43:42.442453 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d20d86-7a73-436e-9b6d-9712cee929ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:43:42.443070 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d20d86-7a73-436e-9b6d-9712cee929ab admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:43:42.443351 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d20d86-7a73-436e-9b6d-9712cee929ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:43:42.443624 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d20d86-7a73-436e-9b6d-9712cee929ab admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:43:42.444287 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d20d86-7a73-436e-9b6d-9712cee929ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:43:42.446220 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d20d86-7a73-436e-9b6d-9712cee929ab admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:43:42.467835 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1243/2474] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:43:42 2022] POST /identity/v3/users => generated 290 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:43:42.471917 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b61f946-797b-462a-b14a-c635c1bf18da admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:43:42.483225 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b61f946-797b-462a-b14a-c635c1bf18da admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:43:42.488137 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfb2326-9d8a-4864-a91c-25d26882caec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:43:42.488956 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfb2326-9d8a-4864-a91c-25d26882caec admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:43:42.489266 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfb2326-9d8a-4864-a91c-25d26882caec admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:43:42.489543 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfb2326-9d8a-4864-a91c-25d26882caec admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:43:42.490235 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfb2326-9d8a-4864-a91c-25d26882caec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:43:42.491814 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfb2326-9d8a-4864-a91c-25d26882caec admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:43:42.502375 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1232/2475] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:43:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:43:42.506174 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d20d86-7a73-436e-9b6d-9712cee929ab admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:43:42.516254 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d20d86-7a73-436e-9b6d-9712cee929ab admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:43:42.520825 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0c41b5-dc72-4fd9-b397-3ffe1938a497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:43:42.521456 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0c41b5-dc72-4fd9-b397-3ffe1938a497 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:43:42.521737 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0c41b5-dc72-4fd9-b397-3ffe1938a497 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:43:42.522009 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0c41b5-dc72-4fd9-b397-3ffe1938a497 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:43:42.522697 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0c41b5-dc72-4fd9-b397-3ffe1938a497 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:43:42.524261 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0c41b5-dc72-4fd9-b397-3ffe1938a497 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:43:42.535659 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1244/2476] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:43:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:43:42.539848 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfb2326-9d8a-4864-a91c-25d26882caec admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:43:42.550421 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfb2326-9d8a-4864-a91c-25d26882caec admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:43:42.555028 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495d2893-9c65-4039-8365-ad0d96f6503e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:43:42.555687 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495d2893-9c65-4039-8365-ad0d96f6503e admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:43:42.555968 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495d2893-9c65-4039-8365-ad0d96f6503e admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:43:42.556244 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495d2893-9c65-4039-8365-ad0d96f6503e admin admin] PATH_INFO: `/v3/projects/94cd3ff87fa1467eadff776897b62f20/users/efb76cf41dde4bb6975ad9210ebd03f8/roles/61b3b93d37a047e4bb8369b46fe20db9` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:43:42.570484 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495d2893-9c65-4039-8365-ad0d96f6503e admin admin] RBAC: Authorizing `identity:create_grant(project_id=94cd3ff87fa1467eadff776897b62f20, user_id=efb76cf41dde4bb6975ad9210ebd03f8, role_id=61b3b93d37a047e4bb8369b46fe20db9)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:43:42.573185 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495d2893-9c65-4039-8365-ad0d96f6503e admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:43:42.587315 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1233/2477] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:43:42 2022] PUT /identity/v3/projects/94cd3ff87fa1467eadff776897b62f20/users/efb76cf41dde4bb6975ad9210ebd03f8/roles/61b3b93d37a047e4bb8369b46fe20db9 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:43:42.599818 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f62e08-db52-4479-8893-bcc66d12b6af None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:43:42.600109 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f62e08-db52-4479-8893-bcc66d12b6af None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:43:42.600383 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f62e08-db52-4479-8893-bcc66d12b6af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:43:42.630439 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-20f62e08-db52-4479-8893-bcc66d12b6af None None] MFA Rules not processed for user `efb76cf41dde4bb6975ad9210ebd03f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:43:42.631798 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-20f62e08-db52-4479-8893-bcc66d12b6af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:43:42.694421 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1245/2478] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:43:42 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:43:42.699140 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495d2893-9c65-4039-8365-ad0d96f6503e admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:43:42.760477 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495d2893-9c65-4039-8365-ad0d96f6503e admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:43:42.765304 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c7b172-9f76-4628-9678-c078d0a2d0b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:43:42.765923 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c7b172-9f76-4628-9678-c078d0a2d0b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:43:42.766206 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c7b172-9f76-4628-9678-c078d0a2d0b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:43:42.766482 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c7b172-9f76-4628-9678-c078d0a2d0b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:43:42.767137 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c7b172-9f76-4628-9678-c078d0a2d0b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:43:42.769026 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c7b172-9f76-4628-9678-c078d0a2d0b7 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:43:42.799013 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1234/2479] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Aug 6 01:43:42 2022] POST /identity/v3/projects => generated 381 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:43:42.800232 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f62e08-db52-4479-8893-bcc66d12b6af None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:43:42.811456 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f62e08-db52-4479-8893-bcc66d12b6af None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:43:42.867243 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fc7aea-c6f0-48df-aed8-c0f94423f0ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:43:42.867853 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fc7aea-c6f0-48df-aed8-c0f94423f0ba admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:43:42.868135 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fc7aea-c6f0-48df-aed8-c0f94423f0ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:43:42.868412 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fc7aea-c6f0-48df-aed8-c0f94423f0ba admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:43:42.869095 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fc7aea-c6f0-48df-aed8-c0f94423f0ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:43:42.871687 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fc7aea-c6f0-48df-aed8-c0f94423f0ba admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:43:42.891180 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1246/2480] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:43:42 2022] POST /identity/v3/users => generated 297 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:43:42.895247 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c7b172-9f76-4628-9678-c078d0a2d0b7 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:43:42.905480 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c7b172-9f76-4628-9678-c078d0a2d0b7 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:43:42.910165 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a019e9-5512-497d-b655-121f8a8bee70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:43:42.910782 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a019e9-5512-497d-b655-121f8a8bee70 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:43:42.911068 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a019e9-5512-497d-b655-121f8a8bee70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:43:42.911350 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a019e9-5512-497d-b655-121f8a8bee70 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:43:42.912077 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a019e9-5512-497d-b655-121f8a8bee70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:43:42.913724 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a019e9-5512-497d-b655-121f8a8bee70 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:43:42.924558 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1235/2481] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:43:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:43:42.928483 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fc7aea-c6f0-48df-aed8-c0f94423f0ba admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:43:42.938933 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fc7aea-c6f0-48df-aed8-c0f94423f0ba admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:43:42.943589 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f79709d-405a-4592-b2de-7022d743a517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:43:42.944263 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f79709d-405a-4592-b2de-7022d743a517 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:43:42.944602 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f79709d-405a-4592-b2de-7022d743a517 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:43:42.944826 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f79709d-405a-4592-b2de-7022d743a517 admin admin] PATH_INFO: `/v3/projects/92fbaa113ed5459d8c2165e1d776c50b/users/99e9f6fea30e46208539a27a3ecd2838/roles/4559b9a0837249f895a13c70096f8d38` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:43:42.947073 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:43:42.961073 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f79709d-405a-4592-b2de-7022d743a517 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92fbaa113ed5459d8c2165e1d776c50b, user_id=99e9f6fea30e46208539a27a3ecd2838, role_id=4559b9a0837249f895a13c70096f8d38)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:43:42.963725 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f79709d-405a-4592-b2de-7022d743a517 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:43:42.977250 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1247/2482] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:43:42 2022] PUT /identity/v3/projects/92fbaa113ed5459d8c2165e1d776c50b/users/99e9f6fea30e46208539a27a3ecd2838/roles/4559b9a0837249f895a13c70096f8d38 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:43:42.981723 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a019e9-5512-497d-b655-121f8a8bee70 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:43:42.991876 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a019e9-5512-497d-b655-121f8a8bee70 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:43:43.048701 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b630f3-9295-44f7-83ec-eb2a13febdcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:43:43.049360 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b630f3-9295-44f7-83ec-eb2a13febdcd admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:43:43.049648 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b630f3-9295-44f7-83ec-eb2a13febdcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:43:43.049928 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b630f3-9295-44f7-83ec-eb2a13febdcd admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:43:43.050717 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b630f3-9295-44f7-83ec-eb2a13febdcd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:43:43.052309 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b630f3-9295-44f7-83ec-eb2a13febdcd admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:43:43.053598 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:43:43.064699 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1236/2483] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:43:42 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:43:43.068710 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f79709d-405a-4592-b2de-7022d743a517 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:43:43.079568 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f79709d-405a-4592-b2de-7022d743a517 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:43:43.084153 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a81feb-cbb3-4066-926a-99f2d3f2af2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:43:43.084812 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a81feb-cbb3-4066-926a-99f2d3f2af2e admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:43:43.085135 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a81feb-cbb3-4066-926a-99f2d3f2af2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:43:43.085413 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a81feb-cbb3-4066-926a-99f2d3f2af2e admin admin] PATH_INFO: `/v3/projects/92fbaa113ed5459d8c2165e1d776c50b/users/99e9f6fea30e46208539a27a3ecd2838/roles/4559b9a0837249f895a13c70096f8d38` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:43:43.087355 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a81feb-cbb3-4066-926a-99f2d3f2af2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=92fbaa113ed5459d8c2165e1d776c50b, user_id=99e9f6fea30e46208539a27a3ecd2838, role_id=4559b9a0837249f895a13c70096f8d38)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:43:43.090793 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a81feb-cbb3-4066-926a-99f2d3f2af2e admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:43:43.100808 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1248/2484] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:43:43 2022] PUT /identity/v3/projects/92fbaa113ed5459d8c2165e1d776c50b/users/99e9f6fea30e46208539a27a3ecd2838/roles/4559b9a0837249f895a13c70096f8d38 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:43:43.111102 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f87d2bd-a895-4e6b-aeb7-d507c01f8293 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:43:43.111388 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f87d2bd-a895-4e6b-aeb7-d507c01f8293 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:43:43.111671 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f87d2bd-a895-4e6b-aeb7-d507c01f8293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:43:43.139677 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-9f87d2bd-a895-4e6b-aeb7-d507c01f8293 None None] MFA Rules not processed for user `99e9f6fea30e46208539a27a3ecd2838`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:43:43.140880 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-9f87d2bd-a895-4e6b-aeb7-d507c01f8293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:43:43.207858 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1237/2485] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:43:43 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:43:43.219273 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a81feb-cbb3-4066-926a-99f2d3f2af2e admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:43:43.221018 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-c6a81feb-cbb3-4066-926a-99f2d3f2af2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:43:43.298826 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a81feb-cbb3-4066-926a-99f2d3f2af2e admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:43:43.302485 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d90cdf3-e9bc-4d91-877c-3622c640b2eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1be1e6173e374a60b9e1fbf51a1ab571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:43:43.303089 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d90cdf3-e9bc-4d91-877c-3622c640b2eb service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:43:43.303369 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d90cdf3-e9bc-4d91-877c-3622c640b2eb service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:43:43.303647 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d90cdf3-e9bc-4d91-877c-3622c640b2eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:43:43.313095 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d90cdf3-e9bc-4d91-877c-3622c640b2eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:43:43.314504 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d90cdf3-e9bc-4d91-877c-3622c640b2eb service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:43:43.324324 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1249/2486] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:43:43 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:43:43.402875 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f87d2bd-a895-4e6b-aeb7-d507c01f8293 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:43:43.412469 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f87d2bd-a895-4e6b-aeb7-d507c01f8293 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:43:43.415991 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007322b6-8a2e-4372-8eb3-dbb0e6b244e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1be1e6173e374a60b9e1fbf51a1ab571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:43:43.416594 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007322b6-8a2e-4372-8eb3-dbb0e6b244e8 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:43:43.416875 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007322b6-8a2e-4372-8eb3-dbb0e6b244e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:43:43.417909 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007322b6-8a2e-4372-8eb3-dbb0e6b244e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:43:43.473978 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007322b6-8a2e-4372-8eb3-dbb0e6b244e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:43:43.475334 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007322b6-8a2e-4372-8eb3-dbb0e6b244e8 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:43:43.484723 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1238/2487] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:43:43 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:43:45.914393 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d90cdf3-e9bc-4d91-877c-3622c640b2eb service cinder] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:43:45.915802 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-4d90cdf3-e9bc-4d91-877c-3622c640b2eb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:43:45.927487 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d90cdf3-e9bc-4d91-877c-3622c640b2eb service cinder] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:43:45.931125 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbca6450-1d50-4615-937e-2ebd23b95257 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1be1e6173e374a60b9e1fbf51a1ab571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:43:45.931731 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbca6450-1d50-4615-937e-2ebd23b95257 service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:43:45.932018 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbca6450-1d50-4615-937e-2ebd23b95257 service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:43:45.932295 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbca6450-1d50-4615-937e-2ebd23b95257 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:43:45.934060 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-dbca6450-1d50-4615-937e-2ebd23b95257 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:43:46.014075 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbca6450-1d50-4615-937e-2ebd23b95257 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:43:46.015441 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbca6450-1d50-4615-937e-2ebd23b95257 service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:43:46.034771 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1250/2488] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:43:45 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:43:53.481388 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=64 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:43:54.442601 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007322b6-8a2e-4372-8eb3-dbb0e6b244e8 service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:43:54.505300 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007322b6-8a2e-4372-8eb3-dbb0e6b244e8 service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:43:54.509967 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c30b19-5531-41f4-b0e1-f58292c45291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:43:54.510607 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c30b19-5531-41f4-b0e1-f58292c45291 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:43:54.510893 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c30b19-5531-41f4-b0e1-f58292c45291 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:43:54.511175 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c30b19-5531-41f4-b0e1-f58292c45291 admin admin] PATH_INFO: `/v3/users/efb76cf41dde4bb6975ad9210ebd03f8` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:43:54.512961 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c30b19-5531-41f4-b0e1-f58292c45291 admin admin] RBAC: Authorizing `identity:delete_user(user_id=efb76cf41dde4bb6975ad9210ebd03f8)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:43:54.514930 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c30b19-5531-41f4-b0e1-f58292c45291 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:43:54.561817 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-68c30b19-5531-41f4-b0e1-f58292c45291 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'efb76cf41dde4bb6975ad9210ebd03f8'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:43:54.562402 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-68c30b19-5531-41f4-b0e1-f58292c45291 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'efb76cf41dde4bb6975ad9210ebd03f8'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:43:54.563030 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-68c30b19-5531-41f4-b0e1-f58292c45291 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'efb76cf41dde4bb6975ad9210ebd03f8'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:43:54.575669 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-68c30b19-5531-41f4-b0e1-f58292c45291 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'efb76cf41dde4bb6975ad9210ebd03f8'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:43:54.582563 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-68c30b19-5531-41f4-b0e1-f58292c45291 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'efb76cf41dde4bb6975ad9210ebd03f8'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:43:54.595397 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1239/2489] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Aug 6 01:43:54 2022] DELETE /identity/v3/users/efb76cf41dde4bb6975ad9210ebd03f8 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:43:54.600019 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f74b31-e49e-4ae1-b5df-243f44a8dab1 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:43:54.600566 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f74b31-e49e-4ae1-b5df-243f44a8dab1 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:43:54.600667 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f74b31-e49e-4ae1-b5df-243f44a8dab1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:43:54.619802 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-c5f74b31-e49e-4ae1-b5df-243f44a8dab1 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:43:54.621303 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-c5f74b31-e49e-4ae1-b5df-243f44a8dab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:43:54.639342 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:43:54.683493 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1251/2490] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:43:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:43:54.693732 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c30b19-5531-41f4-b0e1-f58292c45291 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:43:54.695143 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-68c30b19-5531-41f4-b0e1-f58292c45291 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:43:54.754971 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c30b19-5531-41f4-b0e1-f58292c45291 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:43:54.758718 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650d3c18-eaa5-4f5d-ace0-9d19980b1e6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:43:54.759332 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650d3c18-eaa5-4f5d-ace0-9d19980b1e6a service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:43:54.759619 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650d3c18-eaa5-4f5d-ace0-9d19980b1e6a service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:43:54.759899 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650d3c18-eaa5-4f5d-ace0-9d19980b1e6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:43:54.769738 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650d3c18-eaa5-4f5d-ace0-9d19980b1e6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:43:54.771197 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650d3c18-eaa5-4f5d-ace0-9d19980b1e6a service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:43:54.781589 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1240/2491] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:43:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:43:55.050558 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f74b31-e49e-4ae1-b5df-243f44a8dab1 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:43:55.051988 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-c5f74b31-e49e-4ae1-b5df-243f44a8dab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:43:55.064678 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f74b31-e49e-4ae1-b5df-243f44a8dab1 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:43:55.069561 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48f0a7e-758b-4e79-b6b0-9acbe41726f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:43:55.070217 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48f0a7e-758b-4e79-b6b0-9acbe41726f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:43:55.070497 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48f0a7e-758b-4e79-b6b0-9acbe41726f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:43:55.070839 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48f0a7e-758b-4e79-b6b0-9acbe41726f0 admin admin] PATH_INFO: `/v3/projects/94cd3ff87fa1467eadff776897b62f20` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:43:55.072116 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48f0a7e-758b-4e79-b6b0-9acbe41726f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=94cd3ff87fa1467eadff776897b62f20)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:43:55.074091 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48f0a7e-758b-4e79-b6b0-9acbe41726f0 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:43:55.120827 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-f48f0a7e-758b-4e79-b6b0-9acbe41726f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94cd3ff87fa1467eadff776897b62f20', 'request_id': 'req-f48f0a7e-758b-4e79-b6b0-9acbe41726f0'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:43:55.127424 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-f48f0a7e-758b-4e79-b6b0-9acbe41726f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94cd3ff87fa1467eadff776897b62f20', 'request_id': 'req-f48f0a7e-758b-4e79-b6b0-9acbe41726f0'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:43:55.132862 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-f48f0a7e-758b-4e79-b6b0-9acbe41726f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94cd3ff87fa1467eadff776897b62f20', 'request_id': 'req-f48f0a7e-758b-4e79-b6b0-9acbe41726f0'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:43:55.144714 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-f48f0a7e-758b-4e79-b6b0-9acbe41726f0 admin admin] The token cache is being invalidate because project 94cd3ff87fa1467eadff776897b62f20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:43:55.145004 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-f48f0a7e-758b-4e79-b6b0-9acbe41726f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:43:55.145654 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-f48f0a7e-758b-4e79-b6b0-9acbe41726f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:43:55.147147 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1252/2492] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Aug 6 01:43:55 2022] DELETE /identity/v3/projects/94cd3ff87fa1467eadff776897b62f20 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:43:55.151277 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650d3c18-eaa5-4f5d-ace0-9d19980b1e6a service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:43:55.161354 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650d3c18-eaa5-4f5d-ace0-9d19980b1e6a service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:43:55.180486 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:43:55.225320 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b8e3b8-c62a-44b1-b723-9b5af3bccafb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:43:55.227795 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b8e3b8-c62a-44b1-b723-9b5af3bccafb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:43:55.227795 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b8e3b8-c62a-44b1-b723-9b5af3bccafb admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:43:55.227795 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b8e3b8-c62a-44b1-b723-9b5af3bccafb admin admin] PATH_INFO: `/v3/users/99e9f6fea30e46208539a27a3ecd2838` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:43:55.227795 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b8e3b8-c62a-44b1-b723-9b5af3bccafb admin admin] RBAC: Authorizing `identity:delete_user(user_id=99e9f6fea30e46208539a27a3ecd2838)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:43:55.231702 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b8e3b8-c62a-44b1-b723-9b5af3bccafb admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:43:55.284915 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-37b8e3b8-c62a-44b1-b723-9b5af3bccafb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99e9f6fea30e46208539a27a3ecd2838'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:43:55.285665 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-37b8e3b8-c62a-44b1-b723-9b5af3bccafb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99e9f6fea30e46208539a27a3ecd2838'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:43:55.286343 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-37b8e3b8-c62a-44b1-b723-9b5af3bccafb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99e9f6fea30e46208539a27a3ecd2838'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:43:55.299208 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-37b8e3b8-c62a-44b1-b723-9b5af3bccafb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99e9f6fea30e46208539a27a3ecd2838'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:43:55.306345 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-37b8e3b8-c62a-44b1-b723-9b5af3bccafb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99e9f6fea30e46208539a27a3ecd2838'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:43:55.319935 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1241/2493] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Aug 6 01:43:55 2022] DELETE /identity/v3/users/99e9f6fea30e46208539a27a3ecd2838 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:43:55.605023 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48f0a7e-758b-4e79-b6b0-9acbe41726f0 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:43:55.606660 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-f48f0a7e-758b-4e79-b6b0-9acbe41726f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:43:55.621952 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48f0a7e-758b-4e79-b6b0-9acbe41726f0 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:43:55.686147 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95a5d72-4f0b-4403-a5fe-fd32940d0e2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:43:55.686950 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95a5d72-4f0b-4403-a5fe-fd32940d0e2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:43:55.687237 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95a5d72-4f0b-4403-a5fe-fd32940d0e2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:43:55.687511 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95a5d72-4f0b-4403-a5fe-fd32940d0e2a admin admin] PATH_INFO: `/v3/projects/92fbaa113ed5459d8c2165e1d776c50b` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:43:55.688703 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95a5d72-4f0b-4403-a5fe-fd32940d0e2a admin admin] RBAC: Authorizing `identity:delete_project(project_id=92fbaa113ed5459d8c2165e1d776c50b)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:43:55.690654 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95a5d72-4f0b-4403-a5fe-fd32940d0e2a admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:43:55.737129 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:43:55.738393 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-d95a5d72-4f0b-4403-a5fe-fd32940d0e2a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92fbaa113ed5459d8c2165e1d776c50b', 'request_id': 'req-d95a5d72-4f0b-4403-a5fe-fd32940d0e2a'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:43:55.746000 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-d95a5d72-4f0b-4403-a5fe-fd32940d0e2a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92fbaa113ed5459d8c2165e1d776c50b', 'request_id': 'req-d95a5d72-4f0b-4403-a5fe-fd32940d0e2a'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:43:55.752017 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-d95a5d72-4f0b-4403-a5fe-fd32940d0e2a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92fbaa113ed5459d8c2165e1d776c50b', 'request_id': 'req-d95a5d72-4f0b-4403-a5fe-fd32940d0e2a'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:43:55.765191 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-d95a5d72-4f0b-4403-a5fe-fd32940d0e2a admin admin] The token cache is being invalidate because project 92fbaa113ed5459d8c2165e1d776c50b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:43:55.765494 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-d95a5d72-4f0b-4403-a5fe-fd32940d0e2a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:43:55.766197 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-d95a5d72-4f0b-4403-a5fe-fd32940d0e2a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:43:55.767749 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1253/2494] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Aug 6 01:43:55 2022] DELETE /identity/v3/projects/92fbaa113ed5459d8c2165e1d776c50b => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:43:55.772578 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfdf8b3-8d1f-45f8-a145-e98e4dad5f68 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:43:55.773109 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfdf8b3-8d1f-45f8-a145-e98e4dad5f68 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:43:55.773220 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfdf8b3-8d1f-45f8-a145-e98e4dad5f68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:43:55.792825 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-7dfdf8b3-8d1f-45f8-a145-e98e4dad5f68 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:43:55.794183 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-7dfdf8b3-8d1f-45f8-a145-e98e4dad5f68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:43:55.851655 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1242/2495] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:43:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:43:55.860621 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadec2f2-b882-4807-9a20-96a5cd478ad6 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:43:55.860914 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadec2f2-b882-4807-9a20-96a5cd478ad6 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:43:55.861228 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadec2f2-b882-4807-9a20-96a5cd478ad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:43:55.880796 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-fadec2f2-b882-4807-9a20-96a5cd478ad6 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:43:55.882029 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-fadec2f2-b882-4807-9a20-96a5cd478ad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:43:55.890433 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1254/2496] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:43:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:43:55.894506 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfdf8b3-8d1f-45f8-a145-e98e4dad5f68 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:43:55.904599 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfdf8b3-8d1f-45f8-a145-e98e4dad5f68 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:43:55.909211 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcb2fc8-4151-4247-94a8-91a07e20ecc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:43:55.909817 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcb2fc8-4151-4247-94a8-91a07e20ecc8 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:43:55.910104 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcb2fc8-4151-4247-94a8-91a07e20ecc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:43:55.910379 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcb2fc8-4151-4247-94a8-91a07e20ecc8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:43:55.911071 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcb2fc8-4151-4247-94a8-91a07e20ecc8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:43:55.911377 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcb2fc8-4151-4247-94a8-91a07e20ecc8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:43:55.912953 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcb2fc8-4151-4247-94a8-91a07e20ecc8 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:43:55.924346 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1243/2497] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Aug 6 01:43:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:43:55.928560 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadec2f2-b882-4807-9a20-96a5cd478ad6 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:43:55.938651 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadec2f2-b882-4807-9a20-96a5cd478ad6 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:43:55.943322 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8690f80-1747-470a-b997-7b3ab3c5b977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:43:55.943926 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8690f80-1747-470a-b997-7b3ab3c5b977 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:43:55.944210 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8690f80-1747-470a-b997-7b3ab3c5b977 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:43:55.944488 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8690f80-1747-470a-b997-7b3ab3c5b977 admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:43:55.945181 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8690f80-1747-470a-b997-7b3ab3c5b977 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:43:55.947090 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8690f80-1747-470a-b997-7b3ab3c5b977 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:43:55.977243 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1255/2498] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Aug 6 01:43:55 2022] POST /identity/v3/projects => generated 363 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:43:55.978357 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcb2fc8-4151-4247-94a8-91a07e20ecc8 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:43:55.988360 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcb2fc8-4151-4247-94a8-91a07e20ecc8 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:43:56.015205 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:43:56.048418 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88978c8e-6cff-44e1-8dbb-3a7991f90568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:43:56.049033 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88978c8e-6cff-44e1-8dbb-3a7991f90568 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:43:56.049347 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88978c8e-6cff-44e1-8dbb-3a7991f90568 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:43:56.049628 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88978c8e-6cff-44e1-8dbb-3a7991f90568 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:43:56.050302 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88978c8e-6cff-44e1-8dbb-3a7991f90568 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:43:56.052212 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88978c8e-6cff-44e1-8dbb-3a7991f90568 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:43:56.072068 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1244/2499] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:43:55 2022] POST /identity/v3/users => generated 282 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:43:56.076573 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8690f80-1747-470a-b997-7b3ab3c5b977 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:43:56.086710 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8690f80-1747-470a-b997-7b3ab3c5b977 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:43:56.091876 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5039542d-6452-4bee-b3a9-6b3c6d1d0200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:43:56.092479 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5039542d-6452-4bee-b3a9-6b3c6d1d0200 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:43:56.092784 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5039542d-6452-4bee-b3a9-6b3c6d1d0200 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:43:56.093077 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5039542d-6452-4bee-b3a9-6b3c6d1d0200 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:43:56.093779 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5039542d-6452-4bee-b3a9-6b3c6d1d0200 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:43:56.095344 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5039542d-6452-4bee-b3a9-6b3c6d1d0200 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:43:56.104974 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1256/2500] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:43:56 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:43:56.109403 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88978c8e-6cff-44e1-8dbb-3a7991f90568 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:43:56.119422 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88978c8e-6cff-44e1-8dbb-3a7991f90568 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:43:56.124321 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01af63b-528e-4cff-b20d-04b2c257ada1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:43:56.125289 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01af63b-528e-4cff-b20d-04b2c257ada1 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:43:56.125574 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01af63b-528e-4cff-b20d-04b2c257ada1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:43:56.125850 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01af63b-528e-4cff-b20d-04b2c257ada1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:43:56.126537 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01af63b-528e-4cff-b20d-04b2c257ada1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:43:56.128105 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01af63b-528e-4cff-b20d-04b2c257ada1 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:43:56.138734 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1245/2501] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:43:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:43:56.142838 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5039542d-6452-4bee-b3a9-6b3c6d1d0200 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:43:56.152578 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5039542d-6452-4bee-b3a9-6b3c6d1d0200 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:43:56.157668 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45fe37e-aa09-48aa-898e-19cdbd6084c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:43:56.158317 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45fe37e-aa09-48aa-898e-19cdbd6084c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:43:56.158596 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45fe37e-aa09-48aa-898e-19cdbd6084c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:43:56.158870 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45fe37e-aa09-48aa-898e-19cdbd6084c9 admin admin] PATH_INFO: `/v3/projects/245727bb61134ec7872bb9f5edba6b34/users/73c59574855441b19c8a92e797ea4a48/roles/61b3b93d37a047e4bb8369b46fe20db9` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:43:56.173518 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45fe37e-aa09-48aa-898e-19cdbd6084c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=245727bb61134ec7872bb9f5edba6b34, user_id=73c59574855441b19c8a92e797ea4a48, role_id=61b3b93d37a047e4bb8369b46fe20db9)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:43:56.176167 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45fe37e-aa09-48aa-898e-19cdbd6084c9 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:43:56.189257 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1257/2502] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:43:56 2022] PUT /identity/v3/projects/245727bb61134ec7872bb9f5edba6b34/users/73c59574855441b19c8a92e797ea4a48/roles/61b3b93d37a047e4bb8369b46fe20db9 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:43:56.201068 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b55ded6-56fe-4a0e-b52f-b80b6c6d08e4 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:43:56.201388 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b55ded6-56fe-4a0e-b52f-b80b6c6d08e4 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:43:56.201673 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b55ded6-56fe-4a0e-b52f-b80b6c6d08e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:43:56.229636 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-8b55ded6-56fe-4a0e-b52f-b80b6c6d08e4 None None] MFA Rules not processed for user `73c59574855441b19c8a92e797ea4a48`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:43:56.231501 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-8b55ded6-56fe-4a0e-b52f-b80b6c6d08e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:43:56.295253 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1246/2503] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:43:56 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:43:56.299673 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45fe37e-aa09-48aa-898e-19cdbd6084c9 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:43:56.360317 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45fe37e-aa09-48aa-898e-19cdbd6084c9 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:43:56.365082 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3da2876-2137-4bd6-be14-f4bb97a43730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:43:56.365748 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3da2876-2137-4bd6-be14-f4bb97a43730 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:43:56.366034 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3da2876-2137-4bd6-be14-f4bb97a43730 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:43:56.366314 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3da2876-2137-4bd6-be14-f4bb97a43730 admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:43:56.367016 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3da2876-2137-4bd6-be14-f4bb97a43730 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:43:56.368967 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3da2876-2137-4bd6-be14-f4bb97a43730 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:43:56.397065 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1258/2504] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Aug 6 01:43:56 2022] POST /identity/v3/projects => generated 363 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:43:56.401217 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b55ded6-56fe-4a0e-b52f-b80b6c6d08e4 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:43:56.413152 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b55ded6-56fe-4a0e-b52f-b80b6c6d08e4 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:43:56.474688 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa951b7-7193-4cba-be3a-cebd326df4c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:43:56.475313 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa951b7-7193-4cba-be3a-cebd326df4c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:43:56.475603 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa951b7-7193-4cba-be3a-cebd326df4c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:43:56.475883 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa951b7-7193-4cba-be3a-cebd326df4c4 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:43:56.476555 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa951b7-7193-4cba-be3a-cebd326df4c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:43:56.478628 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa951b7-7193-4cba-be3a-cebd326df4c4 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:43:56.492417 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3da2876-2137-4bd6-be14-f4bb97a43730 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:43:56.494169 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-a3da2876-2137-4bd6-be14-f4bb97a43730 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:43:56.506447 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1247/2505] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:43:56 2022] POST /identity/v3/users => generated 288 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:43:56.510683 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa951b7-7193-4cba-be3a-cebd326df4c4 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:43:56.520240 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa951b7-7193-4cba-be3a-cebd326df4c4 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:43:56.524927 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7605b4f3-333a-49d0-9366-2de36294483d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:43:56.525553 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7605b4f3-333a-49d0-9366-2de36294483d admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:43:56.525838 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7605b4f3-333a-49d0-9366-2de36294483d admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:43:56.526113 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7605b4f3-333a-49d0-9366-2de36294483d admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:43:56.526830 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7605b4f3-333a-49d0-9366-2de36294483d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:43:56.528452 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7605b4f3-333a-49d0-9366-2de36294483d admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:43:56.541492 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1248/2506] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:43:56 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 01:43:56.542585 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7605b4f3-333a-49d0-9366-2de36294483d admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:43:56.553032 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7605b4f3-333a-49d0-9366-2de36294483d admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:43:56.557836 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ac92e8-9e3d-4446-a0a5-9298903abccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:43:56.558539 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ac92e8-9e3d-4446-a0a5-9298903abccf admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:43:56.558675 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3da2876-2137-4bd6-be14-f4bb97a43730 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:43:56.558822 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ac92e8-9e3d-4446-a0a5-9298903abccf admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:43:56.559101 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ac92e8-9e3d-4446-a0a5-9298903abccf admin admin] PATH_INFO: `/v3/projects/26212daeccfb41db8bff22b3335c2fea/users/2d7f18802fe64417afe0b017cf3f9603/roles/4559b9a0837249f895a13c70096f8d38` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:43:56.563030 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65d3ab2-fa35-4599-afc8-76ed6b07230e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:43:56.563928 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65d3ab2-fa35-4599-afc8-76ed6b07230e service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:43:56.564215 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65d3ab2-fa35-4599-afc8-76ed6b07230e service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:43:56.564504 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65d3ab2-fa35-4599-afc8-76ed6b07230e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:43:56.566341 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-f65d3ab2-fa35-4599-afc8-76ed6b07230e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:43:56.572281 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:43:56.575430 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ac92e8-9e3d-4446-a0a5-9298903abccf admin admin] RBAC: Authorizing `identity:create_grant(project_id=26212daeccfb41db8bff22b3335c2fea, user_id=2d7f18802fe64417afe0b017cf3f9603, role_id=4559b9a0837249f895a13c70096f8d38)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:43:56.579065 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65d3ab2-fa35-4599-afc8-76ed6b07230e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:43:56.579727 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ac92e8-9e3d-4446-a0a5-9298903abccf admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:43:56.580504 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65d3ab2-fa35-4599-afc8-76ed6b07230e service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:43:56.586036 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:43:56.592086 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1259/2507] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:43:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 01:43:56.594864 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1249/2508] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:43:56 2022] PUT /identity/v3/projects/26212daeccfb41db8bff22b3335c2fea/users/2d7f18802fe64417afe0b017cf3f9603/roles/4559b9a0837249f895a13c70096f8d38 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:43:56.598398 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65d3ab2-fa35-4599-afc8-76ed6b07230e service neutron] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:43:56.610178 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65d3ab2-fa35-4599-afc8-76ed6b07230e service neutron] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:43:56.667338 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1c9f9c-f032-45ca-9e28-fe3c18d3bd19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:43:56.667966 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1c9f9c-f032-45ca-9e28-fe3c18d3bd19 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:43:56.668249 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1c9f9c-f032-45ca-9e28-fe3c18d3bd19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:43:56.668525 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1c9f9c-f032-45ca-9e28-fe3c18d3bd19 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:43:56.669258 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1c9f9c-f032-45ca-9e28-fe3c18d3bd19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:43:56.672002 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1c9f9c-f032-45ca-9e28-fe3c18d3bd19 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:43:56.681760 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1260/2509] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:43:56 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 01:43:56.686069 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ac92e8-9e3d-4446-a0a5-9298903abccf admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:43:56.697005 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ac92e8-9e3d-4446-a0a5-9298903abccf admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:43:56.701694 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff7fa55-269d-4b29-9a56-346c6c92d27a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:43:56.702384 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff7fa55-269d-4b29-9a56-346c6c92d27a admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:43:56.702651 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff7fa55-269d-4b29-9a56-346c6c92d27a admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:43:56.702929 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff7fa55-269d-4b29-9a56-346c6c92d27a admin admin] PATH_INFO: `/v3/projects/26212daeccfb41db8bff22b3335c2fea/users/2d7f18802fe64417afe0b017cf3f9603/roles/4559b9a0837249f895a13c70096f8d38` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:43:56.705001 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff7fa55-269d-4b29-9a56-346c6c92d27a admin admin] RBAC: Authorizing `identity:create_grant(project_id=26212daeccfb41db8bff22b3335c2fea, user_id=2d7f18802fe64417afe0b017cf3f9603, role_id=4559b9a0837249f895a13c70096f8d38)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:43:56.707712 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff7fa55-269d-4b29-9a56-346c6c92d27a admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:43:56.718739 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1250/2510] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:43:56 2022] PUT /identity/v3/projects/26212daeccfb41db8bff22b3335c2fea/users/2d7f18802fe64417afe0b017cf3f9603/roles/4559b9a0837249f895a13c70096f8d38 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:43:56.728961 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cfbc54-370f-43b4-90da-d4533d8b3db9 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:43:56.729288 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cfbc54-370f-43b4-90da-d4533d8b3db9 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:43:56.729572 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cfbc54-370f-43b4-90da-d4533d8b3db9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:43:56.758522 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-a8cfbc54-370f-43b4-90da-d4533d8b3db9 None None] MFA Rules not processed for user `2d7f18802fe64417afe0b017cf3f9603`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:43:56.759750 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-a8cfbc54-370f-43b4-90da-d4533d8b3db9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:43:56.831561 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1261/2511] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:43:56 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:43:56.844070 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff7fa55-269d-4b29-9a56-346c6c92d27a admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:43:56.846025 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-aff7fa55-269d-4b29-9a56-346c6c92d27a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:43:56.908896 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff7fa55-269d-4b29-9a56-346c6c92d27a admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:43:56.912680 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa220f24-9d7e-456f-b356-3ee70397ebdc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1be1e6173e374a60b9e1fbf51a1ab571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:43:56.913344 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa220f24-9d7e-456f-b356-3ee70397ebdc service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:43:56.913631 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa220f24-9d7e-456f-b356-3ee70397ebdc service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:43:56.913910 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa220f24-9d7e-456f-b356-3ee70397ebdc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:43:56.924783 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa220f24-9d7e-456f-b356-3ee70397ebdc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:43:56.926286 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa220f24-9d7e-456f-b356-3ee70397ebdc service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:43:56.936959 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1251/2512] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:43:56 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:43:57.015618 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cfbc54-370f-43b4-90da-d4533d8b3db9 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:43:57.025842 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cfbc54-370f-43b4-90da-d4533d8b3db9 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:43:57.030093 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0878a63-25dd-40fc-b174-2e929ba1c3e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1be1e6173e374a60b9e1fbf51a1ab571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:43:57.030788 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0878a63-25dd-40fc-b174-2e929ba1c3e8 service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:43:57.031136 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0878a63-25dd-40fc-b174-2e929ba1c3e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:43:57.031480 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0878a63-25dd-40fc-b174-2e929ba1c3e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:43:57.089903 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0878a63-25dd-40fc-b174-2e929ba1c3e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:43:57.091335 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0878a63-25dd-40fc-b174-2e929ba1c3e8 service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:43:57.100931 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1262/2513] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:43:57 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 01:43:59.789231 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa220f24-9d7e-456f-b356-3ee70397ebdc service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:43:59.790710 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-fa220f24-9d7e-456f-b356-3ee70397ebdc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:43:59.861982 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa220f24-9d7e-456f-b356-3ee70397ebdc service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:43:59.866661 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cfdcc8-edff-4021-b4d5-6ab64aea60ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:43:59.867308 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cfdcc8-edff-4021-b4d5-6ab64aea60ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:43:59.867597 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cfdcc8-edff-4021-b4d5-6ab64aea60ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:43:59.867877 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cfdcc8-edff-4021-b4d5-6ab64aea60ac admin admin] PATH_INFO: `/v3/users/42ade772e11a4725928ab12d3bccc4a7` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:43:59.869119 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cfdcc8-edff-4021-b4d5-6ab64aea60ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=42ade772e11a4725928ab12d3bccc4a7)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:43:59.871394 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cfdcc8-edff-4021-b4d5-6ab64aea60ac admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:43:59.929197 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-97cfdcc8-edff-4021-b4d5-6ab64aea60ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42ade772e11a4725928ab12d3bccc4a7'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:43:59.929889 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-97cfdcc8-edff-4021-b4d5-6ab64aea60ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42ade772e11a4725928ab12d3bccc4a7'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:43:59.930524 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-97cfdcc8-edff-4021-b4d5-6ab64aea60ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42ade772e11a4725928ab12d3bccc4a7'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:43:59.935246 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0878a63-25dd-40fc-b174-2e929ba1c3e8 service cinder] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:43:59.945585 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-97cfdcc8-edff-4021-b4d5-6ab64aea60ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42ade772e11a4725928ab12d3bccc4a7'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:43:59.947010 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0878a63-25dd-40fc-b174-2e929ba1c3e8 service cinder] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:43:59.951824 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f4f7de-9da5-4850-afe4-ad1cbd24483d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:43:59.952551 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f4f7de-9da5-4850-afe4-ad1cbd24483d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:43:59.952844 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f4f7de-9da5-4850-afe4-ad1cbd24483d admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:43:59.952967 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-97cfdcc8-edff-4021-b4d5-6ab64aea60ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42ade772e11a4725928ab12d3bccc4a7'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:43:59.953158 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f4f7de-9da5-4850-afe4-ad1cbd24483d admin admin] PATH_INFO: `/v3/users/73c59574855441b19c8a92e797ea4a48` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:43:59.954332 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f4f7de-9da5-4850-afe4-ad1cbd24483d admin admin] RBAC: Authorizing `identity:delete_user(user_id=73c59574855441b19c8a92e797ea4a48)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:43:59.956260 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f4f7de-9da5-4850-afe4-ad1cbd24483d admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:43:59.963249 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:43:59.971732 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1252/2514] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Aug 6 01:43:59 2022] DELETE /identity/v3/users/42ade772e11a4725928ab12d3bccc4a7 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:43:59.977023 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:44:00.013724 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-67f4f7de-9da5-4850-afe4-ad1cbd24483d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73c59574855441b19c8a92e797ea4a48'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:00.020778 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-67f4f7de-9da5-4850-afe4-ad1cbd24483d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73c59574855441b19c8a92e797ea4a48'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:00.021434 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-67f4f7de-9da5-4850-afe4-ad1cbd24483d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73c59574855441b19c8a92e797ea4a48'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:00.022058 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-67f4f7de-9da5-4850-afe4-ad1cbd24483d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73c59574855441b19c8a92e797ea4a48'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:00.035604 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-67f4f7de-9da5-4850-afe4-ad1cbd24483d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73c59574855441b19c8a92e797ea4a48'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:00.049615 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1263/2515] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Aug 6 01:43:59 2022] DELETE /identity/v3/users/73c59574855441b19c8a92e797ea4a48 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:44:00.054078 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb54a65-0f7d-4b24-bad1-c367f6252fd1 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:00.054442 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb54a65-0f7d-4b24-bad1-c367f6252fd1 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:00.054722 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb54a65-0f7d-4b24-bad1-c367f6252fd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:00.073743 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-ebb54a65-0f7d-4b24-bad1-c367f6252fd1 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:44:00.074951 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-ebb54a65-0f7d-4b24-bad1-c367f6252fd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:00.142662 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1253/2516] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:44:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:44:00.152886 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f4f7de-9da5-4850-afe4-ad1cbd24483d admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:00.155800 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-67f4f7de-9da5-4850-afe4-ad1cbd24483d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:00.162209 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb54a65-0f7d-4b24-bad1-c367f6252fd1 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:00.164246 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-ebb54a65-0f7d-4b24-bad1-c367f6252fd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:00.177870 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb54a65-0f7d-4b24-bad1-c367f6252fd1 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:00.183770 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e8fb5b-8b39-4032-bd46-f5a12bc61bda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:00.184151 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e8fb5b-8b39-4032-bd46-f5a12bc61bda admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:00.184466 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e8fb5b-8b39-4032-bd46-f5a12bc61bda admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:00.184740 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e8fb5b-8b39-4032-bd46-f5a12bc61bda admin admin] PATH_INFO: `/v3/projects/16078c76f554419992faa7570fe5aba1` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:00.186021 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e8fb5b-8b39-4032-bd46-f5a12bc61bda admin admin] RBAC: Authorizing `identity:delete_project(project_id=16078c76f554419992faa7570fe5aba1)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:00.187972 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e8fb5b-8b39-4032-bd46-f5a12bc61bda admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:00.221358 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f4f7de-9da5-4850-afe4-ad1cbd24483d admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:00.225448 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f81112-52e8-4eb2-917d-cda8b243090b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:00.226062 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f81112-52e8-4eb2-917d-cda8b243090b service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:00.226341 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f81112-52e8-4eb2-917d-cda8b243090b service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:00.226737 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f81112-52e8-4eb2-917d-cda8b243090b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:00.236272 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f81112-52e8-4eb2-917d-cda8b243090b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:00.237875 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f81112-52e8-4eb2-917d-cda8b243090b service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:00.240373 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-32e8fb5b-8b39-4032-bd46-f5a12bc61bda admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16078c76f554419992faa7570fe5aba1', 'request_id': 'req-32e8fb5b-8b39-4032-bd46-f5a12bc61bda'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:00.246950 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-32e8fb5b-8b39-4032-bd46-f5a12bc61bda admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16078c76f554419992faa7570fe5aba1', 'request_id': 'req-32e8fb5b-8b39-4032-bd46-f5a12bc61bda'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:00.248260 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1264/2517] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:44:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:44:00.252854 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-32e8fb5b-8b39-4032-bd46-f5a12bc61bda admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16078c76f554419992faa7570fe5aba1', 'request_id': 'req-32e8fb5b-8b39-4032-bd46-f5a12bc61bda'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:00.267306 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-32e8fb5b-8b39-4032-bd46-f5a12bc61bda admin admin] The token cache is being invalidate because project 16078c76f554419992faa7570fe5aba1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:44:00.267665 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-32e8fb5b-8b39-4032-bd46-f5a12bc61bda admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:00.268340 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-32e8fb5b-8b39-4032-bd46-f5a12bc61bda admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:00.269905 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1254/2518] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Aug 6 01:44:00 2022] DELETE /identity/v3/projects/16078c76f554419992faa7570fe5aba1 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:44:00.275691 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562d737a-ac31-4e45-a705-2b52b670ad88 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:00.275982 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562d737a-ac31-4e45-a705-2b52b670ad88 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:00.276261 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562d737a-ac31-4e45-a705-2b52b670ad88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:00.297192 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-562d737a-ac31-4e45-a705-2b52b670ad88 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:44:00.298408 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-562d737a-ac31-4e45-a705-2b52b670ad88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:00.362284 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1265/2519] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:44:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:44:00.370731 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbd5883-1cae-4e92-a453-b65e5b11db15 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:00.371022 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbd5883-1cae-4e92-a453-b65e5b11db15 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:00.371308 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbd5883-1cae-4e92-a453-b65e5b11db15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:00.394399 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-0cbd5883-1cae-4e92-a453-b65e5b11db15 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:44:00.395578 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-0cbd5883-1cae-4e92-a453-b65e5b11db15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:00.412009 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1255/2520] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:44:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:44:00.416780 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562d737a-ac31-4e45-a705-2b52b670ad88 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:00.427848 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562d737a-ac31-4e45-a705-2b52b670ad88 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:00.432614 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee8c44e-88eb-4a02-b398-024c911891cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:00.433268 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee8c44e-88eb-4a02-b398-024c911891cc admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:00.433555 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee8c44e-88eb-4a02-b398-024c911891cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:00.433835 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee8c44e-88eb-4a02-b398-024c911891cc admin admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:00.434535 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee8c44e-88eb-4a02-b398-024c911891cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:44:00.434849 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee8c44e-88eb-4a02-b398-024c911891cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:00.437323 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee8c44e-88eb-4a02-b398-024c911891cc admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:00.449243 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1266/2521] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Aug 6 01:44:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:44:00.454219 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbd5883-1cae-4e92-a453-b65e5b11db15 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:00.464100 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbd5883-1cae-4e92-a453-b65e5b11db15 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:00.469214 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8503877e-6c60-424d-85c8-37109b8961d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:00.469863 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8503877e-6c60-424d-85c8-37109b8961d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:00.470149 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8503877e-6c60-424d-85c8-37109b8961d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:00.470426 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8503877e-6c60-424d-85c8-37109b8961d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:00.471112 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8503877e-6c60-424d-85c8-37109b8961d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:00.473011 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8503877e-6c60-424d-85c8-37109b8961d5 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:00.505499 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1256/2522] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Aug 6 01:44:00 2022] POST /identity/v3/projects => generated 387 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:44:00.506666 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee8c44e-88eb-4a02-b398-024c911891cc admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:00.518537 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee8c44e-88eb-4a02-b398-024c911891cc admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:00.539427 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:44:00.541396 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8503877e-6c60-424d-85c8-37109b8961d5 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:00.543258 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-8503877e-6c60-424d-85c8-37109b8961d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:00.588424 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55327a99-6830-485e-93b7-42d995e97a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:00.589064 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55327a99-6830-485e-93b7-42d995e97a41 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:00.589360 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55327a99-6830-485e-93b7-42d995e97a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:00.589634 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55327a99-6830-485e-93b7-42d995e97a41 admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:00.590296 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55327a99-6830-485e-93b7-42d995e97a41 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:00.592258 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55327a99-6830-485e-93b7-42d995e97a41 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:00.610837 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:44:00.615342 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1267/2523] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:44:00 2022] POST /identity/v3/users => generated 294 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:44:00.616943 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8503877e-6c60-424d-85c8-37109b8961d5 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:00.619491 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55327a99-6830-485e-93b7-42d995e97a41 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:00.624726 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2505802-e3ff-45d9-b37d-3db8c18768f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:00.625786 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2505802-e3ff-45d9-b37d-3db8c18768f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:00.626231 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2505802-e3ff-45d9-b37d-3db8c18768f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:00.626882 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2505802-e3ff-45d9-b37d-3db8c18768f3 admin admin] PATH_INFO: `/v3/projects/245727bb61134ec7872bb9f5edba6b34` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:00.628073 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2505802-e3ff-45d9-b37d-3db8c18768f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=245727bb61134ec7872bb9f5edba6b34)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:00.630059 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2505802-e3ff-45d9-b37d-3db8c18768f3 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:00.630725 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55327a99-6830-485e-93b7-42d995e97a41 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:00.635924 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedac842-26fa-4ac7-b48e-8c62ddd57552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:00.636531 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedac842-26fa-4ac7-b48e-8c62ddd57552 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:00.636859 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedac842-26fa-4ac7-b48e-8c62ddd57552 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:00.637164 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedac842-26fa-4ac7-b48e-8c62ddd57552 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:00.637854 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedac842-26fa-4ac7-b48e-8c62ddd57552 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:00.639907 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedac842-26fa-4ac7-b48e-8c62ddd57552 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:00.650649 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1268/2524] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:44:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 01:44:00.655149 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedac842-26fa-4ac7-b48e-8c62ddd57552 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:00.664950 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedac842-26fa-4ac7-b48e-8c62ddd57552 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:00.670489 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c4a667-6ca1-4487-ae1d-1184daf41047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:00.671148 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c4a667-6ca1-4487-ae1d-1184daf41047 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:00.671485 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c4a667-6ca1-4487-ae1d-1184daf41047 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:00.671804 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c4a667-6ca1-4487-ae1d-1184daf41047 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:00.672490 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c4a667-6ca1-4487-ae1d-1184daf41047 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:00.674240 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c4a667-6ca1-4487-ae1d-1184daf41047 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:00.680633 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-e2505802-e3ff-45d9-b37d-3db8c18768f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '245727bb61134ec7872bb9f5edba6b34', 'request_id': 'req-e2505802-e3ff-45d9-b37d-3db8c18768f3'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:00.684044 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1269/2525] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:44:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 01:44:00.687068 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-e2505802-e3ff-45d9-b37d-3db8c18768f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '245727bb61134ec7872bb9f5edba6b34', 'request_id': 'req-e2505802-e3ff-45d9-b37d-3db8c18768f3'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:00.688146 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c4a667-6ca1-4487-ae1d-1184daf41047 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:00.693098 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-e2505802-e3ff-45d9-b37d-3db8c18768f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '245727bb61134ec7872bb9f5edba6b34', 'request_id': 'req-e2505802-e3ff-45d9-b37d-3db8c18768f3'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:00.697513 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c4a667-6ca1-4487-ae1d-1184daf41047 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:00.705620 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-e2505802-e3ff-45d9-b37d-3db8c18768f3 admin admin] The token cache is being invalidate because project 245727bb61134ec7872bb9f5edba6b34 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:44:00.705915 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-e2505802-e3ff-45d9-b37d-3db8c18768f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:00.706553 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-e2505802-e3ff-45d9-b37d-3db8c18768f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:00.708132 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1257/2526] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Aug 6 01:44:00 2022] DELETE /identity/v3/projects/245727bb61134ec7872bb9f5edba6b34 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:44:00.711949 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2505802-e3ff-45d9-b37d-3db8c18768f3 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:00.713405 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-e2505802-e3ff-45d9-b37d-3db8c18768f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:00.756593 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa3e5d3-a2a6-45c0-9616-f65dbc4941a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:00.757285 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa3e5d3-a2a6-45c0-9616-f65dbc4941a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:00.757571 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa3e5d3-a2a6-45c0-9616-f65dbc4941a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:00.757833 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa3e5d3-a2a6-45c0-9616-f65dbc4941a5 admin admin] PATH_INFO: `/v3/projects/1d59390ed02c4a43a9b30006d0401bda/users/5330c43936984e1db1232dc182eb72c0/roles/61b3b93d37a047e4bb8369b46fe20db9` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:00.772939 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa3e5d3-a2a6-45c0-9616-f65dbc4941a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d59390ed02c4a43a9b30006d0401bda, user_id=5330c43936984e1db1232dc182eb72c0, role_id=61b3b93d37a047e4bb8369b46fe20db9)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:00.775603 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa3e5d3-a2a6-45c0-9616-f65dbc4941a5 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:00.777901 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2505802-e3ff-45d9-b37d-3db8c18768f3 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:00.783278 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5059a0d-9759-4999-82a4-8b630227a0eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:00.783910 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5059a0d-9759-4999-82a4-8b630227a0eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:00.784237 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5059a0d-9759-4999-82a4-8b630227a0eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:00.784475 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5059a0d-9759-4999-82a4-8b630227a0eb admin admin] PATH_INFO: `/v3/users/2d7f18802fe64417afe0b017cf3f9603` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:00.785831 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5059a0d-9759-4999-82a4-8b630227a0eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d7f18802fe64417afe0b017cf3f9603)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:00.787964 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5059a0d-9759-4999-82a4-8b630227a0eb admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:00.790251 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1270/2527] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:44:00 2022] PUT /identity/v3/projects/1d59390ed02c4a43a9b30006d0401bda/users/5330c43936984e1db1232dc182eb72c0/roles/61b3b93d37a047e4bb8369b46fe20db9 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:44:00.800285 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d598ccf-98ee-4b49-999f-2c8cf61b74db None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:00.800574 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d598ccf-98ee-4b49-999f-2c8cf61b74db None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:00.800854 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d598ccf-98ee-4b49-999f-2c8cf61b74db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:00.830417 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-1d598ccf-98ee-4b49-999f-2c8cf61b74db None None] MFA Rules not processed for user `5330c43936984e1db1232dc182eb72c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:44:00.831693 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-1d598ccf-98ee-4b49-999f-2c8cf61b74db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:00.837974 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-d5059a0d-9759-4999-82a4-8b630227a0eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d7f18802fe64417afe0b017cf3f9603'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:00.838682 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-d5059a0d-9759-4999-82a4-8b630227a0eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d7f18802fe64417afe0b017cf3f9603'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:00.839342 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-d5059a0d-9759-4999-82a4-8b630227a0eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d7f18802fe64417afe0b017cf3f9603'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:00.852104 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-d5059a0d-9759-4999-82a4-8b630227a0eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d7f18802fe64417afe0b017cf3f9603'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:00.859155 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-d5059a0d-9759-4999-82a4-8b630227a0eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d7f18802fe64417afe0b017cf3f9603'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:00.872477 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1258/2528] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Aug 6 01:44:00 2022] DELETE /identity/v3/users/2d7f18802fe64417afe0b017cf3f9603 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:44:00.897394 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1271/2529] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:44:00 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:44:00.901945 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5059a0d-9759-4999-82a4-8b630227a0eb admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:00.903343 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-d5059a0d-9759-4999-82a4-8b630227a0eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:00.910730 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:44:00.975506 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5059a0d-9759-4999-82a4-8b630227a0eb admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:00.980560 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a3b476-b6d3-49b5-8e9f-175a8ace0e9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:00.981218 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a3b476-b6d3-49b5-8e9f-175a8ace0e9b admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:00.981555 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a3b476-b6d3-49b5-8e9f-175a8ace0e9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:00.981980 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a3b476-b6d3-49b5-8e9f-175a8ace0e9b admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:00.982809 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a3b476-b6d3-49b5-8e9f-175a8ace0e9b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:00.985438 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a3b476-b6d3-49b5-8e9f-175a8ace0e9b admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:01.019376 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1259/2530] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Aug 6 01:44:00 2022] POST /identity/v3/projects => generated 385 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:44:01.022867 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d598ccf-98ee-4b49-999f-2c8cf61b74db None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:01.028751 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:44:01.092838 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d598ccf-98ee-4b49-999f-2c8cf61b74db None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:01.152515 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20efc7e-c0c2-4e2b-91bc-2a6c0be18040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:01.153066 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20efc7e-c0c2-4e2b-91bc-2a6c0be18040 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:01.153366 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20efc7e-c0c2-4e2b-91bc-2a6c0be18040 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:01.153645 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20efc7e-c0c2-4e2b-91bc-2a6c0be18040 admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:01.154999 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20efc7e-c0c2-4e2b-91bc-2a6c0be18040 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:01.156940 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20efc7e-c0c2-4e2b-91bc-2a6c0be18040 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:01.173820 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a3b476-b6d3-49b5-8e9f-175a8ace0e9b admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:01.176038 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-21a3b476-b6d3-49b5-8e9f-175a8ace0e9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:01.182760 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1272/2531] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:44:01 2022] POST /identity/v3/users => generated 299 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:44:01.187185 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20efc7e-c0c2-4e2b-91bc-2a6c0be18040 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:01.189863 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a3b476-b6d3-49b5-8e9f-175a8ace0e9b admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:01.195975 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d559fd08-c68b-44b0-b401-5109b8a0f853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:01.196936 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d559fd08-c68b-44b0-b401-5109b8a0f853 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:01.197307 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d559fd08-c68b-44b0-b401-5109b8a0f853 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:01.197664 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d559fd08-c68b-44b0-b401-5109b8a0f853 admin admin] PATH_INFO: `/v3/projects/26212daeccfb41db8bff22b3335c2fea` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:01.199228 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d559fd08-c68b-44b0-b401-5109b8a0f853 admin admin] RBAC: Authorizing `identity:delete_project(project_id=26212daeccfb41db8bff22b3335c2fea)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:01.202097 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d559fd08-c68b-44b0-b401-5109b8a0f853 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:01.202245 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20efc7e-c0c2-4e2b-91bc-2a6c0be18040 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:01.207382 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda6f7a5-4da4-4d4c-9646-1bb2e2e42c9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:01.208071 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda6f7a5-4da4-4d4c-9646-1bb2e2e42c9f admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:01.208359 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda6f7a5-4da4-4d4c-9646-1bb2e2e42c9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:01.208635 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda6f7a5-4da4-4d4c-9646-1bb2e2e42c9f admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:01.209390 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda6f7a5-4da4-4d4c-9646-1bb2e2e42c9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:01.211290 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda6f7a5-4da4-4d4c-9646-1bb2e2e42c9f admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:01.223858 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1273/2532] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:44:01 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:44:01.227961 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda6f7a5-4da4-4d4c-9646-1bb2e2e42c9f admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:01.239880 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda6f7a5-4da4-4d4c-9646-1bb2e2e42c9f admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:01.244751 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615188d6-d352-48a0-ad20-6fbc41c40564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:01.245769 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615188d6-d352-48a0-ad20-6fbc41c40564 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:01.246053 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615188d6-d352-48a0-ad20-6fbc41c40564 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:01.246331 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615188d6-d352-48a0-ad20-6fbc41c40564 admin admin] PATH_INFO: `/v3/projects/7d83bfcc7200417f9fe87044fe71b72c/users/34e00a11515f48d4accd00c09e51f621/roles/4559b9a0837249f895a13c70096f8d38` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:01.263942 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615188d6-d352-48a0-ad20-6fbc41c40564 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d83bfcc7200417f9fe87044fe71b72c, user_id=34e00a11515f48d4accd00c09e51f621, role_id=4559b9a0837249f895a13c70096f8d38)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:01.266700 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615188d6-d352-48a0-ad20-6fbc41c40564 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:01.281527 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1274/2533] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:44:01 2022] PUT /identity/v3/projects/7d83bfcc7200417f9fe87044fe71b72c/users/34e00a11515f48d4accd00c09e51f621/roles/4559b9a0837249f895a13c70096f8d38 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:44:01.285671 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615188d6-d352-48a0-ad20-6fbc41c40564 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:01.352431 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615188d6-d352-48a0-ad20-6fbc41c40564 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:01.357180 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f2e7a8-60b5-4a45-9e3c-45b5a512e6e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:01.357811 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f2e7a8-60b5-4a45-9e3c-45b5a512e6e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:01.358093 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f2e7a8-60b5-4a45-9e3c-45b5a512e6e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:01.358812 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f2e7a8-60b5-4a45-9e3c-45b5a512e6e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:01.359059 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f2e7a8-60b5-4a45-9e3c-45b5a512e6e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:01.360904 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f2e7a8-60b5-4a45-9e3c-45b5a512e6e2 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:01.366704 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:44:01.373103 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1275/2534] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:44:01 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:44:01.377037 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f2e7a8-60b5-4a45-9e3c-45b5a512e6e2 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:01.384412 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-d559fd08-c68b-44b0-b401-5109b8a0f853 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26212daeccfb41db8bff22b3335c2fea', 'request_id': 'req-d559fd08-c68b-44b0-b401-5109b8a0f853'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:01.386320 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f2e7a8-60b5-4a45-9e3c-45b5a512e6e2 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:01.391677 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-d559fd08-c68b-44b0-b401-5109b8a0f853 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26212daeccfb41db8bff22b3335c2fea', 'request_id': 'req-d559fd08-c68b-44b0-b401-5109b8a0f853'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:01.397668 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-d559fd08-c68b-44b0-b401-5109b8a0f853 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26212daeccfb41db8bff22b3335c2fea', 'request_id': 'req-d559fd08-c68b-44b0-b401-5109b8a0f853'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:01.411575 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-d559fd08-c68b-44b0-b401-5109b8a0f853 admin admin] The token cache is being invalidate because project 26212daeccfb41db8bff22b3335c2fea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:44:01.411809 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-d559fd08-c68b-44b0-b401-5109b8a0f853 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:01.412428 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-d559fd08-c68b-44b0-b401-5109b8a0f853 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:01.414004 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1260/2535] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Aug 6 01:44:01 2022] DELETE /identity/v3/projects/26212daeccfb41db8bff22b3335c2fea => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:44:01.418736 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90150117-e461-4229-8bea-58e6e00b23ce None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:01.419083 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90150117-e461-4229-8bea-58e6e00b23ce None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:01.419370 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90150117-e461-4229-8bea-58e6e00b23ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:01.439906 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-90150117-e461-4229-8bea-58e6e00b23ce None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:44:01.441189 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-90150117-e461-4229-8bea-58e6e00b23ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:01.450608 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb5a1d0-d9ea-46bf-8fea-b3d94709677f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:01.452256 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb5a1d0-d9ea-46bf-8fea-b3d94709677f admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:01.452610 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb5a1d0-d9ea-46bf-8fea-b3d94709677f admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:01.452892 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb5a1d0-d9ea-46bf-8fea-b3d94709677f admin admin] PATH_INFO: `/v3/projects/7d83bfcc7200417f9fe87044fe71b72c/users/34e00a11515f48d4accd00c09e51f621/roles/4559b9a0837249f895a13c70096f8d38` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:01.454889 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb5a1d0-d9ea-46bf-8fea-b3d94709677f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d83bfcc7200417f9fe87044fe71b72c, user_id=34e00a11515f48d4accd00c09e51f621, role_id=4559b9a0837249f895a13c70096f8d38)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:01.457555 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb5a1d0-d9ea-46bf-8fea-b3d94709677f admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:01.468790 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1276/2536] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:44:01 2022] PUT /identity/v3/projects/7d83bfcc7200417f9fe87044fe71b72c/users/34e00a11515f48d4accd00c09e51f621/roles/4559b9a0837249f895a13c70096f8d38 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:44:01.480254 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5754da-0555-488f-8d24-175f4b8e2ca6 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:01.481325 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5754da-0555-488f-8d24-175f4b8e2ca6 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:01.481325 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5754da-0555-488f-8d24-175f4b8e2ca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:01.509179 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1261/2537] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:44:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:44:01.515615 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-0c5754da-0555-488f-8d24-175f4b8e2ca6 None None] MFA Rules not processed for user `34e00a11515f48d4accd00c09e51f621`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:44:01.516569 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0787ee93-26ab-4e00-bc92-aa7ad73d1451 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:01.516838 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-0c5754da-0555-488f-8d24-175f4b8e2ca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:01.516952 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0787ee93-26ab-4e00-bc92-aa7ad73d1451 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:01.517152 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0787ee93-26ab-4e00-bc92-aa7ad73d1451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:01.538120 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-0787ee93-26ab-4e00-bc92-aa7ad73d1451 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:44:01.539318 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-0787ee93-26ab-4e00-bc92-aa7ad73d1451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:01.548171 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:44:01.593680 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1277/2538] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:44:01 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:44:01.602375 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1262/2539] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:44:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:44:01.604758 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5754da-0555-488f-8d24-175f4b8e2ca6 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:01.606147 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-0c5754da-0555-488f-8d24-175f4b8e2ca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:01.606401 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0787ee93-26ab-4e00-bc92-aa7ad73d1451 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:01.616607 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0787ee93-26ab-4e00-bc92-aa7ad73d1451 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:01.621584 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f06838-605e-4485-8900-851a389baa80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:01.622230 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f06838-605e-4485-8900-851a389baa80 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:01.622460 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f06838-605e-4485-8900-851a389baa80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:01.623411 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f06838-605e-4485-8900-851a389baa80 admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:01.624449 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f06838-605e-4485-8900-851a389baa80 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:44:01.624753 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f06838-605e-4485-8900-851a389baa80 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:01.626353 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f06838-605e-4485-8900-851a389baa80 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:01.637784 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1263/2540] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Aug 6 01:44:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:44:01.642225 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f06838-605e-4485-8900-851a389baa80 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:01.652541 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f06838-605e-4485-8900-851a389baa80 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:01.657321 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c5281a-f45f-4fc4-b275-fdf7528f8b38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:01.657935 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c5281a-f45f-4fc4-b275-fdf7528f8b38 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:01.658222 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c5281a-f45f-4fc4-b275-fdf7528f8b38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:01.658618 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c5281a-f45f-4fc4-b275-fdf7528f8b38 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:01.659398 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c5281a-f45f-4fc4-b275-fdf7528f8b38 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:01.661227 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c5281a-f45f-4fc4-b275-fdf7528f8b38 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:01.664251 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5754da-0555-488f-8d24-175f4b8e2ca6 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:01.667850 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b682c121-99b7-4529-bff6-0d675f85a1f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1be1e6173e374a60b9e1fbf51a1ab571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:01.668457 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b682c121-99b7-4529-bff6-0d675f85a1f8 service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:01.668736 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b682c121-99b7-4529-bff6-0d675f85a1f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:01.669006 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b682c121-99b7-4529-bff6-0d675f85a1f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:01.679398 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b682c121-99b7-4529-bff6-0d675f85a1f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:01.681277 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b682c121-99b7-4529-bff6-0d675f85a1f8 service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:01.689755 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1264/2541] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Aug 6 01:44:01 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:44:01.693677 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c5281a-f45f-4fc4-b275-fdf7528f8b38 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:01.694858 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1278/2542] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:44:01 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:44:01.760456 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c5281a-f45f-4fc4-b275-fdf7528f8b38 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:01.761993 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b682c121-99b7-4529-bff6-0d675f85a1f8 service cinder] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:01.764033 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-b682c121-99b7-4529-bff6-0d675f85a1f8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:01.766826 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fcbb2d-88a4-43b2-ad1d-47bef7e9af14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:01.767448 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fcbb2d-88a4-43b2-ad1d-47bef7e9af14 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:01.767728 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fcbb2d-88a4-43b2-ad1d-47bef7e9af14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:01.768001 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fcbb2d-88a4-43b2-ad1d-47bef7e9af14 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:01.768692 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fcbb2d-88a4-43b2-ad1d-47bef7e9af14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:01.770623 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fcbb2d-88a4-43b2-ad1d-47bef7e9af14 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:01.777589 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b682c121-99b7-4529-bff6-0d675f85a1f8 service cinder] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:01.782947 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c35f2d7-21a1-49ae-8474-6fa3c11b6ef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:01.783636 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c35f2d7-21a1-49ae-8474-6fa3c11b6ef7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:01.784023 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c35f2d7-21a1-49ae-8474-6fa3c11b6ef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:01.784420 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c35f2d7-21a1-49ae-8474-6fa3c11b6ef7 admin admin] PATH_INFO: `/v3/users/5330c43936984e1db1232dc182eb72c0` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:01.785884 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c35f2d7-21a1-49ae-8474-6fa3c11b6ef7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5330c43936984e1db1232dc182eb72c0)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:01.788086 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c35f2d7-21a1-49ae-8474-6fa3c11b6ef7 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:01.793210 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1265/2543] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:44:01 2022] POST /identity/v3/users => generated 292 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:44:01.797341 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fcbb2d-88a4-43b2-ad1d-47bef7e9af14 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:01.807282 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fcbb2d-88a4-43b2-ad1d-47bef7e9af14 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:01.810578 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:44:01.816073 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b250bb-dd7e-4837-a616-8e8b479decde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:01.816731 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b250bb-dd7e-4837-a616-8e8b479decde admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:01.817010 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b250bb-dd7e-4837-a616-8e8b479decde admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:01.817318 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b250bb-dd7e-4837-a616-8e8b479decde admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:01.818023 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b250bb-dd7e-4837-a616-8e8b479decde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:01.819761 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b250bb-dd7e-4837-a616-8e8b479decde admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:01.838411 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1266/2544] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:44:01 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 01:44:01.843584 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b250bb-dd7e-4837-a616-8e8b479decde admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:01.849185 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-6c35f2d7-21a1-49ae-8474-6fa3c11b6ef7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5330c43936984e1db1232dc182eb72c0'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:01.855892 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b250bb-dd7e-4837-a616-8e8b479decde admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:01.856458 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-6c35f2d7-21a1-49ae-8474-6fa3c11b6ef7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5330c43936984e1db1232dc182eb72c0'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:01.857209 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-6c35f2d7-21a1-49ae-8474-6fa3c11b6ef7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5330c43936984e1db1232dc182eb72c0'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:01.857832 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-6c35f2d7-21a1-49ae-8474-6fa3c11b6ef7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5330c43936984e1db1232dc182eb72c0'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:01.861209 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19196dda-da83-4c0e-97a0-c97685cbc3de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:01.861847 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19196dda-da83-4c0e-97a0-c97685cbc3de admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:01.862149 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19196dda-da83-4c0e-97a0-c97685cbc3de admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:01.862422 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19196dda-da83-4c0e-97a0-c97685cbc3de admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:01.863109 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19196dda-da83-4c0e-97a0-c97685cbc3de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:01.864670 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19196dda-da83-4c0e-97a0-c97685cbc3de admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:01.870250 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-6c35f2d7-21a1-49ae-8474-6fa3c11b6ef7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5330c43936984e1db1232dc182eb72c0'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:01.875657 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1267/2545] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:44:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 01:44:01.879698 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19196dda-da83-4c0e-97a0-c97685cbc3de admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:01.881405 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-19196dda-da83-4c0e-97a0-c97685cbc3de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:01.884985 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1279/2546] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Aug 6 01:44:01 2022] DELETE /identity/v3/users/5330c43936984e1db1232dc182eb72c0 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:44:01.890279 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fe962d-dd11-4ef9-a8c1-6581f0d84e49 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:01.890571 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fe962d-dd11-4ef9-a8c1-6581f0d84e49 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:01.890851 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fe962d-dd11-4ef9-a8c1-6581f0d84e49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:01.894824 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19196dda-da83-4c0e-97a0-c97685cbc3de admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:01.910443 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-d4fe962d-dd11-4ef9-a8c1-6581f0d84e49 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:44:01.911651 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-d4fe962d-dd11-4ef9-a8c1-6581f0d84e49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:01.942910 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:44:01.957331 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c89a318-ee08-48bb-83cf-42983c14cf7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:01.958000 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c89a318-ee08-48bb-83cf-42983c14cf7e admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:01.958282 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c89a318-ee08-48bb-83cf-42983c14cf7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:01.958581 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c89a318-ee08-48bb-83cf-42983c14cf7e admin admin] PATH_INFO: `/v3/projects/8ae4aa378ea6450c990bc500a6e29950/users/728196d77beb4f8786f62df4eff902b7/roles/61b3b93d37a047e4bb8369b46fe20db9` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:01.973752 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c89a318-ee08-48bb-83cf-42983c14cf7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ae4aa378ea6450c990bc500a6e29950, user_id=728196d77beb4f8786f62df4eff902b7, role_id=61b3b93d37a047e4bb8369b46fe20db9)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:01.974704 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1280/2547] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:44:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:44:01.976456 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c89a318-ee08-48bb-83cf-42983c14cf7e admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:01.985944 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fe962d-dd11-4ef9-a8c1-6581f0d84e49 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:01.987330 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-d4fe962d-dd11-4ef9-a8c1-6581f0d84e49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:01.990703 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1268/2548] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:44:01 2022] PUT /identity/v3/projects/8ae4aa378ea6450c990bc500a6e29950/users/728196d77beb4f8786f62df4eff902b7/roles/61b3b93d37a047e4bb8369b46fe20db9 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:44:02.001932 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26882b6-3ea7-4f8c-889c-2032ed8cb85c None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:02.002227 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26882b6-3ea7-4f8c-889c-2032ed8cb85c None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:02.002515 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26882b6-3ea7-4f8c-889c-2032ed8cb85c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:02.030555 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-e26882b6-3ea7-4f8c-889c-2032ed8cb85c None None] MFA Rules not processed for user `728196d77beb4f8786f62df4eff902b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:44:02.031818 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-e26882b6-3ea7-4f8c-889c-2032ed8cb85c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:02.047344 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fe962d-dd11-4ef9-a8c1-6581f0d84e49 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:02.095978 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1269/2549] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:44:02 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:44:02.096516 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa785c9-769c-434c-b30c-ebc656b72068 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:02.097171 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa785c9-769c-434c-b30c-ebc656b72068 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:02.097483 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa785c9-769c-434c-b30c-ebc656b72068 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:02.097808 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa785c9-769c-434c-b30c-ebc656b72068 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:02.100437 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26882b6-3ea7-4f8c-889c-2032ed8cb85c None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:02.102156 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:44:02.162368 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa785c9-769c-434c-b30c-ebc656b72068 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:02.163800 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26882b6-3ea7-4f8c-889c-2032ed8cb85c None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:02.163925 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa785c9-769c-434c-b30c-ebc656b72068 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:02.168404 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee07f907-493f-41c1-b8d1-114b55e67a30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:02.169017 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee07f907-493f-41c1-b8d1-114b55e67a30 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:02.169325 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee07f907-493f-41c1-b8d1-114b55e67a30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:02.170164 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee07f907-493f-41c1-b8d1-114b55e67a30 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:02.171183 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee07f907-493f-41c1-b8d1-114b55e67a30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:02.173079 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee07f907-493f-41c1-b8d1-114b55e67a30 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:02.174030 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1281/2550] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:44:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 01:44:02.205233 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1270/2551] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Aug 6 01:44:02 2022] POST /identity/v3/projects => generated 381 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:44:02.206327 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa785c9-769c-434c-b30c-ebc656b72068 service neutron] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:02.216703 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa785c9-769c-434c-b30c-ebc656b72068 service neutron] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:02.277089 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cc2865-fc77-4482-b826-f8e38666c322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:02.277747 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cc2865-fc77-4482-b826-f8e38666c322 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:02.278055 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cc2865-fc77-4482-b826-f8e38666c322 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:02.278334 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cc2865-fc77-4482-b826-f8e38666c322 admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:02.279007 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cc2865-fc77-4482-b826-f8e38666c322 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:02.280989 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cc2865-fc77-4482-b826-f8e38666c322 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:02.302867 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1282/2552] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:44:02 2022] POST /identity/v3/users => generated 297 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:44:02.306936 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee07f907-493f-41c1-b8d1-114b55e67a30 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:02.317859 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee07f907-493f-41c1-b8d1-114b55e67a30 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:02.322771 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61827a04-fd1c-49a8-89b2-75baf6b32a98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:02.323389 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61827a04-fd1c-49a8-89b2-75baf6b32a98 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:02.323671 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61827a04-fd1c-49a8-89b2-75baf6b32a98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:02.323948 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61827a04-fd1c-49a8-89b2-75baf6b32a98 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:02.324639 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61827a04-fd1c-49a8-89b2-75baf6b32a98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:02.326803 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61827a04-fd1c-49a8-89b2-75baf6b32a98 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:02.337534 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1271/2553] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:44:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:44:02.342430 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cc2865-fc77-4482-b826-f8e38666c322 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:02.353016 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cc2865-fc77-4482-b826-f8e38666c322 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:02.357864 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e234ef-bbfc-475e-9354-9272097d7878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:02.358531 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e234ef-bbfc-475e-9354-9272097d7878 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:02.358813 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e234ef-bbfc-475e-9354-9272097d7878 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:02.359090 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e234ef-bbfc-475e-9354-9272097d7878 admin admin] PATH_INFO: `/v3/projects/8d1caa761da048b8a9cf16201318143c/users/5caf7485815b4721a5a6fb2aabfed8e9/roles/4559b9a0837249f895a13c70096f8d38` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:02.374886 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e234ef-bbfc-475e-9354-9272097d7878 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d1caa761da048b8a9cf16201318143c, user_id=5caf7485815b4721a5a6fb2aabfed8e9, role_id=4559b9a0837249f895a13c70096f8d38)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:02.377661 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e234ef-bbfc-475e-9354-9272097d7878 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:02.394653 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1283/2554] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:44:02 2022] PUT /identity/v3/projects/8d1caa761da048b8a9cf16201318143c/users/5caf7485815b4721a5a6fb2aabfed8e9/roles/4559b9a0837249f895a13c70096f8d38 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:44:02.395676 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61827a04-fd1c-49a8-89b2-75baf6b32a98 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:02.411068 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61827a04-fd1c-49a8-89b2-75baf6b32a98 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:02.441957 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:44:02.461220 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e234ef-bbfc-475e-9354-9272097d7878 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:02.463254 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-07e234ef-bbfc-475e-9354-9272097d7878 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:02.476941 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0cf2e8-ab81-466c-9c2a-68a65bf4529f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:02.477616 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0cf2e8-ab81-466c-9c2a-68a65bf4529f admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:02.478218 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0cf2e8-ab81-466c-9c2a-68a65bf4529f admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:02.478500 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0cf2e8-ab81-466c-9c2a-68a65bf4529f admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:02.479204 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0cf2e8-ab81-466c-9c2a-68a65bf4529f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:02.481309 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0cf2e8-ab81-466c-9c2a-68a65bf4529f admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:02.503059 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1272/2555] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:44:02 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 01:44:02.507808 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0cf2e8-ab81-466c-9c2a-68a65bf4529f admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:02.519161 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0cf2e8-ab81-466c-9c2a-68a65bf4529f admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:02.524388 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68202e04-d3c8-4420-abce-da57ecf13273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:02.525158 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68202e04-d3c8-4420-abce-da57ecf13273 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:02.525645 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68202e04-d3c8-4420-abce-da57ecf13273 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:02.525930 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68202e04-d3c8-4420-abce-da57ecf13273 admin admin] PATH_INFO: `/v3/projects/8d1caa761da048b8a9cf16201318143c/users/5caf7485815b4721a5a6fb2aabfed8e9/roles/4559b9a0837249f895a13c70096f8d38` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:02.528472 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68202e04-d3c8-4420-abce-da57ecf13273 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d1caa761da048b8a9cf16201318143c, user_id=5caf7485815b4721a5a6fb2aabfed8e9, role_id=4559b9a0837249f895a13c70096f8d38)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:02.531950 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68202e04-d3c8-4420-abce-da57ecf13273 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:02.533942 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e234ef-bbfc-475e-9354-9272097d7878 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:02.538683 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fea005-bb26-45b0-abb5-578a9995abb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:02.539315 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fea005-bb26-45b0-abb5-578a9995abb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:02.539593 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fea005-bb26-45b0-abb5-578a9995abb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:02.539869 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fea005-bb26-45b0-abb5-578a9995abb0 admin admin] PATH_INFO: `/v3/projects/1d59390ed02c4a43a9b30006d0401bda` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:02.541006 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fea005-bb26-45b0-abb5-578a9995abb0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d59390ed02c4a43a9b30006d0401bda)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:02.542576 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1273/2556] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:44:02 2022] PUT /identity/v3/projects/8d1caa761da048b8a9cf16201318143c/users/5caf7485815b4721a5a6fb2aabfed8e9/roles/4559b9a0837249f895a13c70096f8d38 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:44:02.542920 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fea005-bb26-45b0-abb5-578a9995abb0 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:02.552132 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:44:02.554806 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d90e028-376d-45b6-87b9-53872404004a None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:02.555097 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d90e028-376d-45b6-87b9-53872404004a None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:02.555386 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d90e028-376d-45b6-87b9-53872404004a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:02.585265 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-2d90e028-376d-45b6-87b9-53872404004a None None] MFA Rules not processed for user `5caf7485815b4721a5a6fb2aabfed8e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:44:02.586465 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-2d90e028-376d-45b6-87b9-53872404004a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:02.595486 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a1fea005-bb26-45b0-abb5-578a9995abb0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d59390ed02c4a43a9b30006d0401bda', 'request_id': 'req-a1fea005-bb26-45b0-abb5-578a9995abb0'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:02.601686 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a1fea005-bb26-45b0-abb5-578a9995abb0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d59390ed02c4a43a9b30006d0401bda', 'request_id': 'req-a1fea005-bb26-45b0-abb5-578a9995abb0'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:02.608547 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a1fea005-bb26-45b0-abb5-578a9995abb0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d59390ed02c4a43a9b30006d0401bda', 'request_id': 'req-a1fea005-bb26-45b0-abb5-578a9995abb0'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:02.626590 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a1fea005-bb26-45b0-abb5-578a9995abb0 admin admin] The token cache is being invalidate because project 1d59390ed02c4a43a9b30006d0401bda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:44:02.626590 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a1fea005-bb26-45b0-abb5-578a9995abb0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:02.626590 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a1fea005-bb26-45b0-abb5-578a9995abb0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:02.629356 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1284/2557] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Aug 6 01:44:02 2022] DELETE /identity/v3/projects/1d59390ed02c4a43a9b30006d0401bda => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:44:02.633297 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fea005-bb26-45b0-abb5-578a9995abb0 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:02.634788 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-a1fea005-bb26-45b0-abb5-578a9995abb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:02.658563 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1274/2558] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:44:02 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:44:02.668944 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d90e028-376d-45b6-87b9-53872404004a None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:02.671146 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-2d90e028-376d-45b6-87b9-53872404004a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:02.699809 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fea005-bb26-45b0-abb5-578a9995abb0 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:02.704876 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575c13f9-b643-401f-96b6-52340d5cee3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:02.705548 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575c13f9-b643-401f-96b6-52340d5cee3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:02.705829 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575c13f9-b643-401f-96b6-52340d5cee3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:02.706104 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575c13f9-b643-401f-96b6-52340d5cee3f admin admin] PATH_INFO: `/v3/users/34e00a11515f48d4accd00c09e51f621` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:02.707361 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575c13f9-b643-401f-96b6-52340d5cee3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=34e00a11515f48d4accd00c09e51f621)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:02.709276 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575c13f9-b643-401f-96b6-52340d5cee3f admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:02.727929 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d90e028-376d-45b6-87b9-53872404004a None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:02.731607 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ad20de-fa00-4105-9d84-94b7a0c29adb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1be1e6173e374a60b9e1fbf51a1ab571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:02.732942 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ad20de-fa00-4105-9d84-94b7a0c29adb service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:02.733331 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ad20de-fa00-4105-9d84-94b7a0c29adb service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:02.733606 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ad20de-fa00-4105-9d84-94b7a0c29adb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:02.743481 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ad20de-fa00-4105-9d84-94b7a0c29adb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:02.744884 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ad20de-fa00-4105-9d84-94b7a0c29adb service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:02.755614 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1275/2559] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:44:02 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 01:44:02.760235 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-575c13f9-b643-401f-96b6-52340d5cee3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34e00a11515f48d4accd00c09e51f621'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:02.767734 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-575c13f9-b643-401f-96b6-52340d5cee3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34e00a11515f48d4accd00c09e51f621'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:02.768518 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-575c13f9-b643-401f-96b6-52340d5cee3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34e00a11515f48d4accd00c09e51f621'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:02.769246 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-575c13f9-b643-401f-96b6-52340d5cee3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34e00a11515f48d4accd00c09e51f621'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:02.782764 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-575c13f9-b643-401f-96b6-52340d5cee3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34e00a11515f48d4accd00c09e51f621'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:02.797376 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1285/2560] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Aug 6 01:44:02 2022] DELETE /identity/v3/users/34e00a11515f48d4accd00c09e51f621 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:44:02.865702 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ad20de-fa00-4105-9d84-94b7a0c29adb service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:02.875812 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ad20de-fa00-4105-9d84-94b7a0c29adb service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:02.921934 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:44:02.931688 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c912db-a939-4e04-91ff-01b5023a484a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1be1e6173e374a60b9e1fbf51a1ab571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:02.932396 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c912db-a939-4e04-91ff-01b5023a484a service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:02.932681 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c912db-a939-4e04-91ff-01b5023a484a service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:02.932989 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c912db-a939-4e04-91ff-01b5023a484a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:02.935026 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-f5c912db-a939-4e04-91ff-01b5023a484a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:03.004906 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c912db-a939-4e04-91ff-01b5023a484a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:03.006394 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c912db-a939-4e04-91ff-01b5023a484a service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:03.016435 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1276/2561] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:44:02 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:44:03.083854 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575c13f9-b643-401f-96b6-52340d5cee3f admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:03.085558 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-575c13f9-b643-401f-96b6-52340d5cee3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:03.111076 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:44:03.173272 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575c13f9-b643-401f-96b6-52340d5cee3f admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:03.178868 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50175e56-4e08-4762-877a-8496b5558612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:03.179522 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50175e56-4e08-4762-877a-8496b5558612 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:03.179802 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50175e56-4e08-4762-877a-8496b5558612 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:03.180080 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50175e56-4e08-4762-877a-8496b5558612 admin admin] PATH_INFO: `/v3/projects/7d83bfcc7200417f9fe87044fe71b72c` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:03.181304 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50175e56-4e08-4762-877a-8496b5558612 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d83bfcc7200417f9fe87044fe71b72c)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:03.183239 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50175e56-4e08-4762-877a-8496b5558612 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:03.230846 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-50175e56-4e08-4762-877a-8496b5558612 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d83bfcc7200417f9fe87044fe71b72c', 'request_id': 'req-50175e56-4e08-4762-877a-8496b5558612'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:03.237547 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-50175e56-4e08-4762-877a-8496b5558612 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d83bfcc7200417f9fe87044fe71b72c', 'request_id': 'req-50175e56-4e08-4762-877a-8496b5558612'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:03.244639 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-50175e56-4e08-4762-877a-8496b5558612 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d83bfcc7200417f9fe87044fe71b72c', 'request_id': 'req-50175e56-4e08-4762-877a-8496b5558612'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:03.257931 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-50175e56-4e08-4762-877a-8496b5558612 admin admin] The token cache is being invalidate because project 7d83bfcc7200417f9fe87044fe71b72c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:44:03.258227 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-50175e56-4e08-4762-877a-8496b5558612 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:03.258850 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-50175e56-4e08-4762-877a-8496b5558612 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:03.260715 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1286/2562] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Aug 6 01:44:03 2022] DELETE /identity/v3/projects/7d83bfcc7200417f9fe87044fe71b72c => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:44:03.266651 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608cb24b-c246-47fa-9fc0-239d47e91451 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:03.266946 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608cb24b-c246-47fa-9fc0-239d47e91451 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:03.267228 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608cb24b-c246-47fa-9fc0-239d47e91451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:03.289107 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-608cb24b-c246-47fa-9fc0-239d47e91451 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:44:03.290490 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-608cb24b-c246-47fa-9fc0-239d47e91451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:03.375571 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1277/2563] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:44:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:44:03.377577 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd13a4d6-7c64-48f4-a5a0-8325ed237b2d None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:03.377858 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd13a4d6-7c64-48f4-a5a0-8325ed237b2d None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:03.378138 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd13a4d6-7c64-48f4-a5a0-8325ed237b2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:03.402400 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-bd13a4d6-7c64-48f4-a5a0-8325ed237b2d None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:44:03.403685 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-bd13a4d6-7c64-48f4-a5a0-8325ed237b2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:03.413984 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1287/2564] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:44:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:44:03.418396 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608cb24b-c246-47fa-9fc0-239d47e91451 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:03.428637 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608cb24b-c246-47fa-9fc0-239d47e91451 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:03.433259 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41519ff5-1998-47b2-8fb1-64b188b06740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:03.433866 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41519ff5-1998-47b2-8fb1-64b188b06740 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:03.434147 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41519ff5-1998-47b2-8fb1-64b188b06740 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:03.434425 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41519ff5-1998-47b2-8fb1-64b188b06740 admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:03.435168 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41519ff5-1998-47b2-8fb1-64b188b06740 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:44:03.435468 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41519ff5-1998-47b2-8fb1-64b188b06740 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:03.437040 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41519ff5-1998-47b2-8fb1-64b188b06740 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:03.448383 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1278/2565] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Aug 6 01:44:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:44:03.452808 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd13a4d6-7c64-48f4-a5a0-8325ed237b2d None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:03.462245 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd13a4d6-7c64-48f4-a5a0-8325ed237b2d None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:03.467579 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919d7b99-b2fd-4445-9ec8-d9937d76f087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:03.468151 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919d7b99-b2fd-4445-9ec8-d9937d76f087 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:03.468432 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919d7b99-b2fd-4445-9ec8-d9937d76f087 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:03.468704 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919d7b99-b2fd-4445-9ec8-d9937d76f087 admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:03.469404 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919d7b99-b2fd-4445-9ec8-d9937d76f087 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:03.471304 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919d7b99-b2fd-4445-9ec8-d9937d76f087 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:03.498418 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1288/2566] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Aug 6 01:44:03 2022] POST /identity/v3/projects => generated 379 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:44:03.502593 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41519ff5-1998-47b2-8fb1-64b188b06740 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:03.512783 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41519ff5-1998-47b2-8fb1-64b188b06740 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:03.569899 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dba92ac-b38c-49db-b229-84f8d1e34d25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:03.570507 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dba92ac-b38c-49db-b229-84f8d1e34d25 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:03.570786 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dba92ac-b38c-49db-b229-84f8d1e34d25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:03.571061 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dba92ac-b38c-49db-b229-84f8d1e34d25 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:03.571725 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dba92ac-b38c-49db-b229-84f8d1e34d25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:03.573651 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dba92ac-b38c-49db-b229-84f8d1e34d25 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:03.594934 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1279/2567] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:44:03 2022] POST /identity/v3/users => generated 290 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:44:03.598966 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919d7b99-b2fd-4445-9ec8-d9937d76f087 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:03.610249 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919d7b99-b2fd-4445-9ec8-d9937d76f087 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:03.614961 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141efbd6-a6a9-47ca-a7b1-2375ebb164f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:03.615575 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141efbd6-a6a9-47ca-a7b1-2375ebb164f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:03.615861 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141efbd6-a6a9-47ca-a7b1-2375ebb164f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:03.616138 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141efbd6-a6a9-47ca-a7b1-2375ebb164f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:03.616838 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141efbd6-a6a9-47ca-a7b1-2375ebb164f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:03.618657 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141efbd6-a6a9-47ca-a7b1-2375ebb164f6 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:03.631870 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1289/2568] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:44:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:44:03.632994 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dba92ac-b38c-49db-b229-84f8d1e34d25 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:03.642957 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dba92ac-b38c-49db-b229-84f8d1e34d25 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:03.647747 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8104da4b-9fdd-4fd0-8e5f-651d64f0dbbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:03.648365 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8104da4b-9fdd-4fd0-8e5f-651d64f0dbbf admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:03.648650 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8104da4b-9fdd-4fd0-8e5f-651d64f0dbbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:03.648930 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8104da4b-9fdd-4fd0-8e5f-651d64f0dbbf admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:03.649656 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8104da4b-9fdd-4fd0-8e5f-651d64f0dbbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:03.651247 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8104da4b-9fdd-4fd0-8e5f-651d64f0dbbf admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:03.661791 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1280/2569] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:44:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:44:03.665880 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141efbd6-a6a9-47ca-a7b1-2375ebb164f6 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:03.676654 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141efbd6-a6a9-47ca-a7b1-2375ebb164f6 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:03.681522 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1285358b-1421-4b3c-9f0a-a00f9d4e812e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:03.682201 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1285358b-1421-4b3c-9f0a-a00f9d4e812e admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:03.682479 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1285358b-1421-4b3c-9f0a-a00f9d4e812e admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:03.682750 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1285358b-1421-4b3c-9f0a-a00f9d4e812e admin admin] PATH_INFO: `/v3/projects/d004b77ebdd3487f96091b9e969b9c18/users/d57d450575cf4bb8b01ad2b6828d2886/roles/61b3b93d37a047e4bb8369b46fe20db9` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:03.697690 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1285358b-1421-4b3c-9f0a-a00f9d4e812e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d004b77ebdd3487f96091b9e969b9c18, user_id=d57d450575cf4bb8b01ad2b6828d2886, role_id=61b3b93d37a047e4bb8369b46fe20db9)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:03.700453 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1285358b-1421-4b3c-9f0a-a00f9d4e812e admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:03.714398 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1290/2570] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:44:03 2022] PUT /identity/v3/projects/d004b77ebdd3487f96091b9e969b9c18/users/d57d450575cf4bb8b01ad2b6828d2886/roles/61b3b93d37a047e4bb8369b46fe20db9 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:44:03.725760 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30398d2-9782-4db8-98ae-57ea2d196db5 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:03.725989 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30398d2-9782-4db8-98ae-57ea2d196db5 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:03.726271 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30398d2-9782-4db8-98ae-57ea2d196db5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:03.734525 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:44:03.755373 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-f30398d2-9782-4db8-98ae-57ea2d196db5 None None] MFA Rules not processed for user `d57d450575cf4bb8b01ad2b6828d2886`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:44:03.756590 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-f30398d2-9782-4db8-98ae-57ea2d196db5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:03.819480 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1281/2571] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:44:03 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:44:03.824077 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1285358b-1421-4b3c-9f0a-a00f9d4e812e admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:03.869659 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:44:03.889156 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1285358b-1421-4b3c-9f0a-a00f9d4e812e admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:03.893858 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2665a8f-2562-4547-87db-04993dcb852b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:03.894473 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2665a8f-2562-4547-87db-04993dcb852b admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:03.894755 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2665a8f-2562-4547-87db-04993dcb852b admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:03.895059 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2665a8f-2562-4547-87db-04993dcb852b admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:03.895720 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2665a8f-2562-4547-87db-04993dcb852b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:03.897665 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2665a8f-2562-4547-87db-04993dcb852b admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:03.923804 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1291/2572] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Aug 6 01:44:03 2022] POST /identity/v3/projects => generated 379 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:44:03.927850 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30398d2-9782-4db8-98ae-57ea2d196db5 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:03.938321 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30398d2-9782-4db8-98ae-57ea2d196db5 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:03.993035 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db8da3d-9099-4581-8bad-20922c3ba52e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:03.993602 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db8da3d-9099-4581-8bad-20922c3ba52e admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:03.993887 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db8da3d-9099-4581-8bad-20922c3ba52e admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:03.994164 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db8da3d-9099-4581-8bad-20922c3ba52e admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:03.994828 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db8da3d-9099-4581-8bad-20922c3ba52e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:03.996993 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db8da3d-9099-4581-8bad-20922c3ba52e admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:04.019175 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1282/2573] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:44:03 2022] POST /identity/v3/users => generated 290 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:44:04.023321 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2665a8f-2562-4547-87db-04993dcb852b admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:04.034245 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2665a8f-2562-4547-87db-04993dcb852b admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:04.038864 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeed7e0-7ffe-4d48-b7bc-80e88d064984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:04.039471 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeed7e0-7ffe-4d48-b7bc-80e88d064984 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:04.039752 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeed7e0-7ffe-4d48-b7bc-80e88d064984 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:04.040031 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeed7e0-7ffe-4d48-b7bc-80e88d064984 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:04.040724 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aeed7e0-7ffe-4d48-b7bc-80e88d064984 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:04.042547 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aeed7e0-7ffe-4d48-b7bc-80e88d064984 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:04.053551 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1292/2574] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:44:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:44:04.057804 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db8da3d-9099-4581-8bad-20922c3ba52e admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:04.068012 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db8da3d-9099-4581-8bad-20922c3ba52e admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:04.072828 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a2b589-a33f-4000-9d9b-d2c24cbb65f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:04.073465 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a2b589-a33f-4000-9d9b-d2c24cbb65f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:04.073749 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a2b589-a33f-4000-9d9b-d2c24cbb65f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:04.074027 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a2b589-a33f-4000-9d9b-d2c24cbb65f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:04.074718 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a2b589-a33f-4000-9d9b-d2c24cbb65f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:04.076910 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a2b589-a33f-4000-9d9b-d2c24cbb65f6 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:04.086658 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1283/2575] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:44:04 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:44:04.090640 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeed7e0-7ffe-4d48-b7bc-80e88d064984 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:04.100837 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeed7e0-7ffe-4d48-b7bc-80e88d064984 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:04.105481 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bbd287-d111-4ee0-ae8c-bd92250712ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:04.106130 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bbd287-d111-4ee0-ae8c-bd92250712ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:04.106415 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bbd287-d111-4ee0-ae8c-bd92250712ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:04.106691 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bbd287-d111-4ee0-ae8c-bd92250712ff admin admin] PATH_INFO: `/v3/projects/76f10ec322d04039817e415ff6d7b326/users/509a9e84238948f68cc6a4181f7cda46/roles/61b3b93d37a047e4bb8369b46fe20db9` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:04.122347 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bbd287-d111-4ee0-ae8c-bd92250712ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=76f10ec322d04039817e415ff6d7b326, user_id=509a9e84238948f68cc6a4181f7cda46, role_id=61b3b93d37a047e4bb8369b46fe20db9)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:04.125778 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bbd287-d111-4ee0-ae8c-bd92250712ff admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:04.138024 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1293/2576] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:44:04 2022] PUT /identity/v3/projects/76f10ec322d04039817e415ff6d7b326/users/509a9e84238948f68cc6a4181f7cda46/roles/61b3b93d37a047e4bb8369b46fe20db9 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:44:04.149226 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96419b3f-049f-4dcf-b514-b4ca204bffef None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:04.149630 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96419b3f-049f-4dcf-b514-b4ca204bffef None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:04.149914 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96419b3f-049f-4dcf-b514-b4ca204bffef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:04.180690 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-96419b3f-049f-4dcf-b514-b4ca204bffef None None] MFA Rules not processed for user `509a9e84238948f68cc6a4181f7cda46`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:44:04.181921 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-96419b3f-049f-4dcf-b514-b4ca204bffef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:04.248769 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1284/2577] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:44:04 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:44:04.253186 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bbd287-d111-4ee0-ae8c-bd92250712ff admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:04.314035 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bbd287-d111-4ee0-ae8c-bd92250712ff admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:04.318646 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06937c8c-cb34-4e0e-b9d7-3d5bedbd7f41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:04.319250 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06937c8c-cb34-4e0e-b9d7-3d5bedbd7f41 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:04.319527 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06937c8c-cb34-4e0e-b9d7-3d5bedbd7f41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:04.319822 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06937c8c-cb34-4e0e-b9d7-3d5bedbd7f41 admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:04.320477 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06937c8c-cb34-4e0e-b9d7-3d5bedbd7f41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:04.322427 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06937c8c-cb34-4e0e-b9d7-3d5bedbd7f41 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:04.353573 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1294/2578] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Aug 6 01:44:04 2022] POST /identity/v3/projects => generated 377 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:44:04.357737 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96419b3f-049f-4dcf-b514-b4ca204bffef None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:04.363380 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:44:04.371046 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96419b3f-049f-4dcf-b514-b4ca204bffef None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:04.433485 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd2ea68-b935-437b-95a4-86d531d57bb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:04.434094 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd2ea68-b935-437b-95a4-86d531d57bb3 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:04.434374 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd2ea68-b935-437b-95a4-86d531d57bb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:04.434655 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd2ea68-b935-437b-95a4-86d531d57bb3 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:04.435870 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd2ea68-b935-437b-95a4-86d531d57bb3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:04.437866 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd2ea68-b935-437b-95a4-86d531d57bb3 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:04.459858 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1285/2579] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:44:04 2022] POST /identity/v3/users => generated 295 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:44:04.463970 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06937c8c-cb34-4e0e-b9d7-3d5bedbd7f41 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:04.474957 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06937c8c-cb34-4e0e-b9d7-3d5bedbd7f41 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:04.479814 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755252e2-07bf-4a9e-ae4a-ce4c7c0974c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:04.480498 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755252e2-07bf-4a9e-ae4a-ce4c7c0974c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:04.480849 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755252e2-07bf-4a9e-ae4a-ce4c7c0974c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:04.481171 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755252e2-07bf-4a9e-ae4a-ce4c7c0974c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:04.482459 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755252e2-07bf-4a9e-ae4a-ce4c7c0974c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:04.484049 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755252e2-07bf-4a9e-ae4a-ce4c7c0974c9 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:04.494987 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1295/2580] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:44:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:44:04.500002 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd2ea68-b935-437b-95a4-86d531d57bb3 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:04.510184 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd2ea68-b935-437b-95a4-86d531d57bb3 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:04.515791 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d0ef24-9ba4-468a-8a80-723e07664365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:04.516547 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d0ef24-9ba4-468a-8a80-723e07664365 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:04.516839 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d0ef24-9ba4-468a-8a80-723e07664365 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:04.517146 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d0ef24-9ba4-468a-8a80-723e07664365 admin admin] PATH_INFO: `/v3/projects/ffb5f1b7010642d0ad57752aecde8756/users/9635fc61a5234788906dda89c67bba85/roles/4559b9a0837249f895a13c70096f8d38` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:04.532812 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d0ef24-9ba4-468a-8a80-723e07664365 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffb5f1b7010642d0ad57752aecde8756, user_id=9635fc61a5234788906dda89c67bba85, role_id=4559b9a0837249f895a13c70096f8d38)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:04.535719 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d0ef24-9ba4-468a-8a80-723e07664365 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:04.549601 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1286/2581] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:44:04 2022] PUT /identity/v3/projects/ffb5f1b7010642d0ad57752aecde8756/users/9635fc61a5234788906dda89c67bba85/roles/4559b9a0837249f895a13c70096f8d38 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:44:04.553631 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755252e2-07bf-4a9e-ae4a-ce4c7c0974c9 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:04.564936 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755252e2-07bf-4a9e-ae4a-ce4c7c0974c9 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:04.625231 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e92109-a43a-4b61-9175-b7267c3cee43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:04.625864 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e92109-a43a-4b61-9175-b7267c3cee43 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:04.626130 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e92109-a43a-4b61-9175-b7267c3cee43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:04.626407 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e92109-a43a-4b61-9175-b7267c3cee43 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:04.627099 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e92109-a43a-4b61-9175-b7267c3cee43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:04.628724 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e92109-a43a-4b61-9175-b7267c3cee43 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:04.638500 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1296/2582] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:44:04 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:44:04.642699 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d0ef24-9ba4-468a-8a80-723e07664365 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:04.652884 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d0ef24-9ba4-468a-8a80-723e07664365 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:04.657612 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa448156-7055-49ad-98d0-994c3fdd2eef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:04.658268 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa448156-7055-49ad-98d0-994c3fdd2eef admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:04.658550 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa448156-7055-49ad-98d0-994c3fdd2eef admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:04.658824 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa448156-7055-49ad-98d0-994c3fdd2eef admin admin] PATH_INFO: `/v3/projects/ffb5f1b7010642d0ad57752aecde8756/users/9635fc61a5234788906dda89c67bba85/roles/4559b9a0837249f895a13c70096f8d38` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:04.660781 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa448156-7055-49ad-98d0-994c3fdd2eef admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffb5f1b7010642d0ad57752aecde8756, user_id=9635fc61a5234788906dda89c67bba85, role_id=4559b9a0837249f895a13c70096f8d38)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:04.663442 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa448156-7055-49ad-98d0-994c3fdd2eef admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:04.674552 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1287/2583] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:44:04 2022] PUT /identity/v3/projects/ffb5f1b7010642d0ad57752aecde8756/users/9635fc61a5234788906dda89c67bba85/roles/4559b9a0837249f895a13c70096f8d38 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:44:04.684778 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e73ad0-8fea-4735-9ab6-3b984689d2a0 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:04.685723 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e73ad0-8fea-4735-9ab6-3b984689d2a0 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:04.686224 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e73ad0-8fea-4735-9ab6-3b984689d2a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:04.693438 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:44:04.715994 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-96e73ad0-8fea-4735-9ab6-3b984689d2a0 None None] MFA Rules not processed for user `9635fc61a5234788906dda89c67bba85`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:44:04.717462 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-96e73ad0-8fea-4735-9ab6-3b984689d2a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:04.790699 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1297/2584] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:44:04 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:44:04.803311 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa448156-7055-49ad-98d0-994c3fdd2eef admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:04.805130 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-fa448156-7055-49ad-98d0-994c3fdd2eef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:04.878373 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa448156-7055-49ad-98d0-994c3fdd2eef admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:04.882084 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f953d3e7-662a-48a9-852e-0c901121779b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1be1e6173e374a60b9e1fbf51a1ab571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:04.882704 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f953d3e7-662a-48a9-852e-0c901121779b service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:04.882982 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f953d3e7-662a-48a9-852e-0c901121779b service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:04.883249 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f953d3e7-662a-48a9-852e-0c901121779b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:04.892617 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f953d3e7-662a-48a9-852e-0c901121779b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:04.894073 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f953d3e7-662a-48a9-852e-0c901121779b service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:04.903525 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1288/2585] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:44:04 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:44:04.950673 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e73ad0-8fea-4735-9ab6-3b984689d2a0 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:04.952213 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-96e73ad0-8fea-4735-9ab6-3b984689d2a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:04.964784 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e73ad0-8fea-4735-9ab6-3b984689d2a0 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:04.969310 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c646878-bafc-4142-9f4a-b9fe02b7259d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1be1e6173e374a60b9e1fbf51a1ab571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:04.970067 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c646878-bafc-4142-9f4a-b9fe02b7259d service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:04.970442 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c646878-bafc-4142-9f4a-b9fe02b7259d service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:04.970789 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c646878-bafc-4142-9f4a-b9fe02b7259d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:05.030449 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c646878-bafc-4142-9f4a-b9fe02b7259d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:05.031853 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c646878-bafc-4142-9f4a-b9fe02b7259d service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:05.041031 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1298/2586] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:44:04 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:44:05.648254 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f953d3e7-662a-48a9-852e-0c901121779b service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:05.649681 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-f953d3e7-662a-48a9-852e-0c901121779b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:05.709877 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:44:05.713971 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f953d3e7-662a-48a9-852e-0c901121779b service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:05.719985 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8781f416-ec48-4439-b818-eb7c2f7cc7cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:05.720624 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8781f416-ec48-4439-b818-eb7c2f7cc7cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:05.720909 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8781f416-ec48-4439-b818-eb7c2f7cc7cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:05.721219 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8781f416-ec48-4439-b818-eb7c2f7cc7cc admin admin] PATH_INFO: `/v3/users/728196d77beb4f8786f62df4eff902b7` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:05.722368 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8781f416-ec48-4439-b818-eb7c2f7cc7cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=728196d77beb4f8786f62df4eff902b7)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:05.724312 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8781f416-ec48-4439-b818-eb7c2f7cc7cc admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:05.769537 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-8781f416-ec48-4439-b818-eb7c2f7cc7cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '728196d77beb4f8786f62df4eff902b7'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:05.770196 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-8781f416-ec48-4439-b818-eb7c2f7cc7cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '728196d77beb4f8786f62df4eff902b7'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:05.770803 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-8781f416-ec48-4439-b818-eb7c2f7cc7cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '728196d77beb4f8786f62df4eff902b7'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:05.782386 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-8781f416-ec48-4439-b818-eb7c2f7cc7cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '728196d77beb4f8786f62df4eff902b7'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:05.789021 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-8781f416-ec48-4439-b818-eb7c2f7cc7cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '728196d77beb4f8786f62df4eff902b7'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:05.801360 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1289/2587] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Aug 6 01:44:05 2022] DELETE /identity/v3/users/728196d77beb4f8786f62df4eff902b7 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:44:05.805857 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ddccb0-24ef-430f-904e-8961c2323a58 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:05.806329 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ddccb0-24ef-430f-904e-8961c2323a58 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:05.806447 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ddccb0-24ef-430f-904e-8961c2323a58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:05.826512 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-61ddccb0-24ef-430f-904e-8961c2323a58 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:44:05.827724 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-61ddccb0-24ef-430f-904e-8961c2323a58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:05.885592 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1299/2588] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:44:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:44:05.895780 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8781f416-ec48-4439-b818-eb7c2f7cc7cc admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:05.897261 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-8781f416-ec48-4439-b818-eb7c2f7cc7cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:05.955058 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8781f416-ec48-4439-b818-eb7c2f7cc7cc admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:05.958727 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375a99a7-7ffd-44da-8fbc-f4b23e84605d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:05.959343 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375a99a7-7ffd-44da-8fbc-f4b23e84605d service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:05.959621 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375a99a7-7ffd-44da-8fbc-f4b23e84605d service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:05.959894 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375a99a7-7ffd-44da-8fbc-f4b23e84605d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:05.968656 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375a99a7-7ffd-44da-8fbc-f4b23e84605d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:05.970115 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375a99a7-7ffd-44da-8fbc-f4b23e84605d service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:05.979226 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1290/2589] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:44:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:44:06.255863 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ddccb0-24ef-430f-904e-8961c2323a58 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:06.257358 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-61ddccb0-24ef-430f-904e-8961c2323a58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:06.271574 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ddccb0-24ef-430f-904e-8961c2323a58 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:06.276391 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a49b4c-c8dc-4eb7-8974-82970a7de8dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:06.277075 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a49b4c-c8dc-4eb7-8974-82970a7de8dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:06.277405 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a49b4c-c8dc-4eb7-8974-82970a7de8dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:06.277697 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a49b4c-c8dc-4eb7-8974-82970a7de8dc admin admin] PATH_INFO: `/v3/projects/8ae4aa378ea6450c990bc500a6e29950` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:06.278973 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a49b4c-c8dc-4eb7-8974-82970a7de8dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ae4aa378ea6450c990bc500a6e29950)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:06.280803 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a49b4c-c8dc-4eb7-8974-82970a7de8dc admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:06.305457 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:44:06.329085 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-67a49b4c-c8dc-4eb7-8974-82970a7de8dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ae4aa378ea6450c990bc500a6e29950', 'request_id': 'req-67a49b4c-c8dc-4eb7-8974-82970a7de8dc'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:06.335159 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-67a49b4c-c8dc-4eb7-8974-82970a7de8dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ae4aa378ea6450c990bc500a6e29950', 'request_id': 'req-67a49b4c-c8dc-4eb7-8974-82970a7de8dc'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:06.341001 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-67a49b4c-c8dc-4eb7-8974-82970a7de8dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ae4aa378ea6450c990bc500a6e29950', 'request_id': 'req-67a49b4c-c8dc-4eb7-8974-82970a7de8dc'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:06.352876 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-67a49b4c-c8dc-4eb7-8974-82970a7de8dc admin admin] The token cache is being invalidate because project 8ae4aa378ea6450c990bc500a6e29950 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:44:06.353207 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-67a49b4c-c8dc-4eb7-8974-82970a7de8dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:06.353820 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-67a49b4c-c8dc-4eb7-8974-82970a7de8dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:06.355346 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1300/2590] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Aug 6 01:44:06 2022] DELETE /identity/v3/projects/8ae4aa378ea6450c990bc500a6e29950 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:44:06.360110 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375a99a7-7ffd-44da-8fbc-f4b23e84605d service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:06.369656 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375a99a7-7ffd-44da-8fbc-f4b23e84605d service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:06.430041 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defd95af-547a-46c1-a459-44b128e2031b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:06.430801 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defd95af-547a-46c1-a459-44b128e2031b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:06.431159 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defd95af-547a-46c1-a459-44b128e2031b admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:06.431500 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defd95af-547a-46c1-a459-44b128e2031b admin admin] PATH_INFO: `/v3/users/5caf7485815b4721a5a6fb2aabfed8e9` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:06.432675 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defd95af-547a-46c1-a459-44b128e2031b admin admin] RBAC: Authorizing `identity:delete_user(user_id=5caf7485815b4721a5a6fb2aabfed8e9)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:06.434623 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defd95af-547a-46c1-a459-44b128e2031b admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:06.458727 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:44:06.490785 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-defd95af-547a-46c1-a459-44b128e2031b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5caf7485815b4721a5a6fb2aabfed8e9'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:06.491778 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-defd95af-547a-46c1-a459-44b128e2031b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5caf7485815b4721a5a6fb2aabfed8e9'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:06.492502 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-defd95af-547a-46c1-a459-44b128e2031b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5caf7485815b4721a5a6fb2aabfed8e9'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:06.505539 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-defd95af-547a-46c1-a459-44b128e2031b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5caf7485815b4721a5a6fb2aabfed8e9'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:06.513548 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-defd95af-547a-46c1-a459-44b128e2031b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5caf7485815b4721a5a6fb2aabfed8e9'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:06.526480 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1291/2591] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Aug 6 01:44:06 2022] DELETE /identity/v3/users/5caf7485815b4721a5a6fb2aabfed8e9 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:44:06.811529 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a49b4c-c8dc-4eb7-8974-82970a7de8dc admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:06.813206 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-67a49b4c-c8dc-4eb7-8974-82970a7de8dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:06.832864 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a49b4c-c8dc-4eb7-8974-82970a7de8dc admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:06.896601 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35ff875-10aa-4676-b566-1cb8cd7b3e99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:06.897274 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35ff875-10aa-4676-b566-1cb8cd7b3e99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:06.897561 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35ff875-10aa-4676-b566-1cb8cd7b3e99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:06.897840 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35ff875-10aa-4676-b566-1cb8cd7b3e99 admin admin] PATH_INFO: `/v3/projects/8d1caa761da048b8a9cf16201318143c` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:06.899003 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35ff875-10aa-4676-b566-1cb8cd7b3e99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d1caa761da048b8a9cf16201318143c)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:06.900930 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35ff875-10aa-4676-b566-1cb8cd7b3e99 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:06.947272 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-b35ff875-10aa-4676-b566-1cb8cd7b3e99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d1caa761da048b8a9cf16201318143c', 'request_id': 'req-b35ff875-10aa-4676-b566-1cb8cd7b3e99'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:06.953469 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-b35ff875-10aa-4676-b566-1cb8cd7b3e99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d1caa761da048b8a9cf16201318143c', 'request_id': 'req-b35ff875-10aa-4676-b566-1cb8cd7b3e99'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:06.958979 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-b35ff875-10aa-4676-b566-1cb8cd7b3e99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d1caa761da048b8a9cf16201318143c', 'request_id': 'req-b35ff875-10aa-4676-b566-1cb8cd7b3e99'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:06.971997 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-b35ff875-10aa-4676-b566-1cb8cd7b3e99 admin admin] The token cache is being invalidate because project 8d1caa761da048b8a9cf16201318143c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:44:06.972293 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-b35ff875-10aa-4676-b566-1cb8cd7b3e99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:06.972931 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-b35ff875-10aa-4676-b566-1cb8cd7b3e99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:06.974515 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1301/2592] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Aug 6 01:44:06 2022] DELETE /identity/v3/projects/8d1caa761da048b8a9cf16201318143c => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:44:06.979381 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d02c88c-b35a-449d-8b25-0727003d38cf None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:06.979826 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d02c88c-b35a-449d-8b25-0727003d38cf None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:06.979943 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d02c88c-b35a-449d-8b25-0727003d38cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:07.000499 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-1d02c88c-b35a-449d-8b25-0727003d38cf None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:44:07.001728 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-1d02c88c-b35a-449d-8b25-0727003d38cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:07.061605 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1292/2593] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:44:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:44:07.069490 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598f29a2-b654-4576-871f-dc795d7cb731 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:07.069781 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598f29a2-b654-4576-871f-dc795d7cb731 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:07.070062 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598f29a2-b654-4576-871f-dc795d7cb731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:07.091070 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-598f29a2-b654-4576-871f-dc795d7cb731 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:44:07.092584 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-598f29a2-b654-4576-871f-dc795d7cb731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:07.101947 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1302/2594] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:44:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:44:07.106343 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d02c88c-b35a-449d-8b25-0727003d38cf None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:07.116497 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d02c88c-b35a-449d-8b25-0727003d38cf None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:07.121498 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daeefa62-a8d2-420c-8ecc-bef1471b8abb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:07.122206 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeefa62-a8d2-420c-8ecc-bef1471b8abb admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:07.123259 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeefa62-a8d2-420c-8ecc-bef1471b8abb admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:07.123607 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeefa62-a8d2-420c-8ecc-bef1471b8abb admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:07.124445 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daeefa62-a8d2-420c-8ecc-bef1471b8abb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:44:07.124765 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daeefa62-a8d2-420c-8ecc-bef1471b8abb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:07.126406 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daeefa62-a8d2-420c-8ecc-bef1471b8abb admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:07.140002 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1293/2595] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Aug 6 01:44:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:44:07.144426 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598f29a2-b654-4576-871f-dc795d7cb731 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:07.154587 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598f29a2-b654-4576-871f-dc795d7cb731 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:07.159513 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f2479a-7cad-430e-97e5-6c58b39e5c24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:07.160170 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f2479a-7cad-430e-97e5-6c58b39e5c24 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:07.160459 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f2479a-7cad-430e-97e5-6c58b39e5c24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:07.160738 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f2479a-7cad-430e-97e5-6c58b39e5c24 admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:07.161444 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f2479a-7cad-430e-97e5-6c58b39e5c24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:07.163365 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f2479a-7cad-430e-97e5-6c58b39e5c24 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:07.191371 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1303/2596] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Aug 6 01:44:07 2022] POST /identity/v3/projects => generated 387 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:44:07.195795 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daeefa62-a8d2-420c-8ecc-bef1471b8abb admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:07.206713 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daeefa62-a8d2-420c-8ecc-bef1471b8abb admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:07.262942 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c62191-98e6-4695-bb86-ea680ec90bb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:07.263679 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c62191-98e6-4695-bb86-ea680ec90bb2 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:07.263955 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c62191-98e6-4695-bb86-ea680ec90bb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:07.264233 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c62191-98e6-4695-bb86-ea680ec90bb2 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:07.264895 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c62191-98e6-4695-bb86-ea680ec90bb2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:07.266823 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c62191-98e6-4695-bb86-ea680ec90bb2 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:07.287456 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1294/2597] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:44:07 2022] POST /identity/v3/users => generated 294 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:44:07.291601 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f2479a-7cad-430e-97e5-6c58b39e5c24 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:07.302528 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f2479a-7cad-430e-97e5-6c58b39e5c24 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:07.307182 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bfbdf5-11ef-4c5e-9565-da17ab6a1bba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:07.307786 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bfbdf5-11ef-4c5e-9565-da17ab6a1bba admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:07.308068 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bfbdf5-11ef-4c5e-9565-da17ab6a1bba admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:07.308343 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bfbdf5-11ef-4c5e-9565-da17ab6a1bba admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:07.309036 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bfbdf5-11ef-4c5e-9565-da17ab6a1bba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:07.310773 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bfbdf5-11ef-4c5e-9565-da17ab6a1bba admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:07.315870 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:44:07.321325 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1304/2598] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:44:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:44:07.325150 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c62191-98e6-4695-bb86-ea680ec90bb2 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:07.335484 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c62191-98e6-4695-bb86-ea680ec90bb2 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:07.340149 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98c61a5-e5a8-4279-b6d4-e9a27e5487ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:07.340759 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98c61a5-e5a8-4279-b6d4-e9a27e5487ab admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:07.341060 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98c61a5-e5a8-4279-b6d4-e9a27e5487ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:07.341910 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98c61a5-e5a8-4279-b6d4-e9a27e5487ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:07.342599 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98c61a5-e5a8-4279-b6d4-e9a27e5487ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:07.344173 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98c61a5-e5a8-4279-b6d4-e9a27e5487ab admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:07.354602 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1295/2599] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:44:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:44:07.359295 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bfbdf5-11ef-4c5e-9565-da17ab6a1bba admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:07.369259 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bfbdf5-11ef-4c5e-9565-da17ab6a1bba admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:07.374862 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6064724b-137d-432b-976e-b65999a93518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:07.375553 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6064724b-137d-432b-976e-b65999a93518 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:07.375844 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6064724b-137d-432b-976e-b65999a93518 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:07.376152 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6064724b-137d-432b-976e-b65999a93518 admin admin] PATH_INFO: `/v3/projects/0dff934cf5b7463b801c8b879eeb41cc/users/0f9e079cdb6247aa9c0ac2106ad0597a/roles/61b3b93d37a047e4bb8369b46fe20db9` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:07.391431 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6064724b-137d-432b-976e-b65999a93518 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0dff934cf5b7463b801c8b879eeb41cc, user_id=0f9e079cdb6247aa9c0ac2106ad0597a, role_id=61b3b93d37a047e4bb8369b46fe20db9)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:07.394150 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6064724b-137d-432b-976e-b65999a93518 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:07.407796 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1305/2600] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:44:07 2022] PUT /identity/v3/projects/0dff934cf5b7463b801c8b879eeb41cc/users/0f9e079cdb6247aa9c0ac2106ad0597a/roles/61b3b93d37a047e4bb8369b46fe20db9 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:44:07.418040 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdaca20-2679-4053-9ace-7616b4278f3d None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:07.418361 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdaca20-2679-4053-9ace-7616b4278f3d None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:07.418640 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdaca20-2679-4053-9ace-7616b4278f3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:07.426957 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:44:07.448198 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-ebdaca20-2679-4053-9ace-7616b4278f3d None None] MFA Rules not processed for user `0f9e079cdb6247aa9c0ac2106ad0597a`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:44:07.449460 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-ebdaca20-2679-4053-9ace-7616b4278f3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:07.512594 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1296/2601] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:44:07 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:44:07.517137 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6064724b-137d-432b-976e-b65999a93518 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:07.579030 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6064724b-137d-432b-976e-b65999a93518 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:07.583805 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5920889-b1e6-46c2-867d-2be66f14d882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:07.584417 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5920889-b1e6-46c2-867d-2be66f14d882 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:07.584705 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5920889-b1e6-46c2-867d-2be66f14d882 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:07.584982 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5920889-b1e6-46c2-867d-2be66f14d882 admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:07.585682 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5920889-b1e6-46c2-867d-2be66f14d882 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:07.587587 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5920889-b1e6-46c2-867d-2be66f14d882 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:07.613664 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1306/2602] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Aug 6 01:44:07 2022] POST /identity/v3/projects => generated 389 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:44:07.617950 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdaca20-2679-4053-9ace-7616b4278f3d None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:07.631438 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdaca20-2679-4053-9ace-7616b4278f3d None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:07.687710 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b9c45a-3973-4c70-9986-65ecf220a17f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:07.688325 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b9c45a-3973-4c70-9986-65ecf220a17f admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:07.688608 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b9c45a-3973-4c70-9986-65ecf220a17f admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:07.688885 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b9c45a-3973-4c70-9986-65ecf220a17f admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:07.689592 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b9c45a-3973-4c70-9986-65ecf220a17f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:07.691504 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b9c45a-3973-4c70-9986-65ecf220a17f admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:07.711749 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1297/2603] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:44:07 2022] POST /identity/v3/users => generated 301 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:44:07.715850 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5920889-b1e6-46c2-867d-2be66f14d882 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:07.726179 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5920889-b1e6-46c2-867d-2be66f14d882 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:07.730799 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a07830-924c-47b7-b42c-a904df4864a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:07.731412 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a07830-924c-47b7-b42c-a904df4864a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:07.731696 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a07830-924c-47b7-b42c-a904df4864a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:07.732910 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a07830-924c-47b7-b42c-a904df4864a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:07.733641 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a07830-924c-47b7-b42c-a904df4864a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:07.735236 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a07830-924c-47b7-b42c-a904df4864a3 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:07.745791 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1307/2604] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:44:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 01:44:07.750782 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b9c45a-3973-4c70-9986-65ecf220a17f admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:07.760569 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b9c45a-3973-4c70-9986-65ecf220a17f admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:07.765384 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4531397-4411-47f1-b642-2905d4839e24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:07.765981 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4531397-4411-47f1-b642-2905d4839e24 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:07.766264 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4531397-4411-47f1-b642-2905d4839e24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:07.766544 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4531397-4411-47f1-b642-2905d4839e24 admin admin] PATH_INFO: `/v3/projects/7cc4b0c35764426284320c4922333ae7/users/f69d83c138984fbc9cdc574b99c9f773/roles/4559b9a0837249f895a13c70096f8d38` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:07.780932 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4531397-4411-47f1-b642-2905d4839e24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7cc4b0c35764426284320c4922333ae7, user_id=f69d83c138984fbc9cdc574b99c9f773, role_id=4559b9a0837249f895a13c70096f8d38)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:07.783596 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4531397-4411-47f1-b642-2905d4839e24 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:07.797068 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1298/2605] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:44:07 2022] PUT /identity/v3/projects/7cc4b0c35764426284320c4922333ae7/users/f69d83c138984fbc9cdc574b99c9f773/roles/4559b9a0837249f895a13c70096f8d38 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:44:07.800963 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a07830-924c-47b7-b42c-a904df4864a3 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:07.811615 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a07830-924c-47b7-b42c-a904df4864a3 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:07.867121 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9d3d32-5984-4270-8d10-0c89d90a291a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:07.867808 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9d3d32-5984-4270-8d10-0c89d90a291a admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:07.868095 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9d3d32-5984-4270-8d10-0c89d90a291a admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:07.868373 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9d3d32-5984-4270-8d10-0c89d90a291a admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:07.869090 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9d3d32-5984-4270-8d10-0c89d90a291a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:07.870981 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9d3d32-5984-4270-8d10-0c89d90a291a admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:07.882069 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1308/2606] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:44:07 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:44:07.886129 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4531397-4411-47f1-b642-2905d4839e24 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:07.897244 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4531397-4411-47f1-b642-2905d4839e24 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:07.902383 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d28522-b5fb-40a4-a31b-d9eeec8f61bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:07.903039 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d28522-b5fb-40a4-a31b-d9eeec8f61bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:07.903328 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d28522-b5fb-40a4-a31b-d9eeec8f61bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:07.904313 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d28522-b5fb-40a4-a31b-d9eeec8f61bb admin admin] PATH_INFO: `/v3/projects/7cc4b0c35764426284320c4922333ae7/users/f69d83c138984fbc9cdc574b99c9f773/roles/4559b9a0837249f895a13c70096f8d38` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:07.906421 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d28522-b5fb-40a4-a31b-d9eeec8f61bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=7cc4b0c35764426284320c4922333ae7, user_id=f69d83c138984fbc9cdc574b99c9f773, role_id=4559b9a0837249f895a13c70096f8d38)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:07.909153 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d28522-b5fb-40a4-a31b-d9eeec8f61bb admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:07.921274 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1299/2607] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:44:07 2022] PUT /identity/v3/projects/7cc4b0c35764426284320c4922333ae7/users/f69d83c138984fbc9cdc574b99c9f773/roles/4559b9a0837249f895a13c70096f8d38 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:44:07.932237 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b2f96a-bfeb-49b0-89c4-d32f2ba28d10 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:07.932524 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b2f96a-bfeb-49b0-89c4-d32f2ba28d10 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:07.932800 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b2f96a-bfeb-49b0-89c4-d32f2ba28d10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:07.965282 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-81b2f96a-bfeb-49b0-89c4-d32f2ba28d10 None None] MFA Rules not processed for user `f69d83c138984fbc9cdc574b99c9f773`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:44:07.967043 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-81b2f96a-bfeb-49b0-89c4-d32f2ba28d10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:07.980027 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d28522-b5fb-40a4-a31b-d9eeec8f61bb admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:07.982533 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-a5d28522-b5fb-40a4-a31b-d9eeec8f61bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:08.002903 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:44:08.035323 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:44:08.040340 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1309/2608] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:44:07 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:44:08.051875 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b2f96a-bfeb-49b0-89c4-d32f2ba28d10 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:08.053394 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-81b2f96a-bfeb-49b0-89c4-d32f2ba28d10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:08.056120 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d28522-b5fb-40a4-a31b-d9eeec8f61bb admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:08.061253 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc2d7bd-8092-4f63-ab41-7e03f9cf99ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:08.061848 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc2d7bd-8092-4f63-ab41-7e03f9cf99ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:08.062031 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc2d7bd-8092-4f63-ab41-7e03f9cf99ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:08.062313 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc2d7bd-8092-4f63-ab41-7e03f9cf99ba admin admin] PATH_INFO: `/v3/users/d57d450575cf4bb8b01ad2b6828d2886` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:08.063522 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc2d7bd-8092-4f63-ab41-7e03f9cf99ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=d57d450575cf4bb8b01ad2b6828d2886)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:08.065485 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc2d7bd-8092-4f63-ab41-7e03f9cf99ba admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:08.114439 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-ddc2d7bd-8092-4f63-ab41-7e03f9cf99ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd57d450575cf4bb8b01ad2b6828d2886'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:08.115228 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-ddc2d7bd-8092-4f63-ab41-7e03f9cf99ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd57d450575cf4bb8b01ad2b6828d2886'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:08.115858 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-ddc2d7bd-8092-4f63-ab41-7e03f9cf99ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd57d450575cf4bb8b01ad2b6828d2886'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:08.117008 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b2f96a-bfeb-49b0-89c4-d32f2ba28d10 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:08.121129 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de91ba65-9f87-4c19-becd-90903e2be2a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1be1e6173e374a60b9e1fbf51a1ab571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:08.121775 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de91ba65-9f87-4c19-becd-90903e2be2a9 service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:08.122052 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de91ba65-9f87-4c19-becd-90903e2be2a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:08.122317 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de91ba65-9f87-4c19-becd-90903e2be2a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:08.126119 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-de91ba65-9f87-4c19-becd-90903e2be2a9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:08.129215 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-ddc2d7bd-8092-4f63-ab41-7e03f9cf99ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd57d450575cf4bb8b01ad2b6828d2886'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:08.136414 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-ddc2d7bd-8092-4f63-ab41-7e03f9cf99ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd57d450575cf4bb8b01ad2b6828d2886'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:08.139653 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de91ba65-9f87-4c19-becd-90903e2be2a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:08.141208 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de91ba65-9f87-4c19-becd-90903e2be2a9 service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:08.150579 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1300/2609] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Aug 6 01:44:07 2022] DELETE /identity/v3/users/d57d450575cf4bb8b01ad2b6828d2886 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:44:08.151012 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1310/2610] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:44:08 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 01:44:08.155400 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300ef229-3375-41c7-8c15-3496c5911072 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:08.155783 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300ef229-3375-41c7-8c15-3496c5911072 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:08.156152 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300ef229-3375-41c7-8c15-3496c5911072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:08.174980 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-300ef229-3375-41c7-8c15-3496c5911072 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:44:08.176193 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-300ef229-3375-41c7-8c15-3496c5911072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:08.241784 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1301/2611] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:44:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:44:08.252605 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de91ba65-9f87-4c19-becd-90903e2be2a9 service cinder] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:08.254131 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-de91ba65-9f87-4c19-becd-90903e2be2a9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:08.297361 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300ef229-3375-41c7-8c15-3496c5911072 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:08.298791 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-300ef229-3375-41c7-8c15-3496c5911072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:08.311260 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300ef229-3375-41c7-8c15-3496c5911072 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:08.312538 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de91ba65-9f87-4c19-becd-90903e2be2a9 service cinder] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:08.316332 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d31535-d81a-4dc0-af7b-ad8fa6897bfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:08.316966 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d31535-d81a-4dc0-af7b-ad8fa6897bfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:08.317286 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d31535-d81a-4dc0-af7b-ad8fa6897bfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:08.317567 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d31535-d81a-4dc0-af7b-ad8fa6897bfc admin admin] PATH_INFO: `/v3/users/0f9e079cdb6247aa9c0ac2106ad0597a` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:08.318745 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d31535-d81a-4dc0-af7b-ad8fa6897bfc admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f9e079cdb6247aa9c0ac2106ad0597a)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:08.320657 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d31535-d81a-4dc0-af7b-ad8fa6897bfc admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:08.364234 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f050890a-4850-4379-afd4-c50c50b3a514 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:08.364860 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f050890a-4850-4379-afd4-c50c50b3a514 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:08.365242 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f050890a-4850-4379-afd4-c50c50b3a514 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:08.365521 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f050890a-4850-4379-afd4-c50c50b3a514 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:08.368973 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-86d31535-d81a-4dc0-af7b-ad8fa6897bfc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f9e079cdb6247aa9c0ac2106ad0597a'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:08.369738 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-86d31535-d81a-4dc0-af7b-ad8fa6897bfc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f9e079cdb6247aa9c0ac2106ad0597a'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:08.370383 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-86d31535-d81a-4dc0-af7b-ad8fa6897bfc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f9e079cdb6247aa9c0ac2106ad0597a'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:08.375522 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f050890a-4850-4379-afd4-c50c50b3a514 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:08.376984 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f050890a-4850-4379-afd4-c50c50b3a514 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:08.380954 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:44:08.383270 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-86d31535-d81a-4dc0-af7b-ad8fa6897bfc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f9e079cdb6247aa9c0ac2106ad0597a'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:08.387094 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1311/2612] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:44:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:44:08.393982 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-86d31535-d81a-4dc0-af7b-ad8fa6897bfc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f9e079cdb6247aa9c0ac2106ad0597a'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:08.411080 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1302/2613] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Aug 6 01:44:08 2022] DELETE /identity/v3/users/0f9e079cdb6247aa9c0ac2106ad0597a => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:44:08.415645 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49313026-8945-4b17-a8c9-7aeaec527553 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:08.415933 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49313026-8945-4b17-a8c9-7aeaec527553 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:08.416205 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49313026-8945-4b17-a8c9-7aeaec527553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:08.436421 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-49313026-8945-4b17-a8c9-7aeaec527553 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:44:08.437669 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-49313026-8945-4b17-a8c9-7aeaec527553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:08.469645 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:44:08.502871 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1312/2614] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:44:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:44:08.512411 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d31535-d81a-4dc0-af7b-ad8fa6897bfc admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:08.514818 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-86d31535-d81a-4dc0-af7b-ad8fa6897bfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:08.578062 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d31535-d81a-4dc0-af7b-ad8fa6897bfc admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:08.581804 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc356b7e-537b-48a5-8696-5fd0b89bf063 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:08.582435 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc356b7e-537b-48a5-8696-5fd0b89bf063 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:08.582718 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc356b7e-537b-48a5-8696-5fd0b89bf063 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:08.582994 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc356b7e-537b-48a5-8696-5fd0b89bf063 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:08.592591 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc356b7e-537b-48a5-8696-5fd0b89bf063 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:08.594044 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc356b7e-537b-48a5-8696-5fd0b89bf063 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:08.603769 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1303/2615] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:44:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:44:08.715606 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49313026-8945-4b17-a8c9-7aeaec527553 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:08.717795 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-49313026-8945-4b17-a8c9-7aeaec527553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:08.730615 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49313026-8945-4b17-a8c9-7aeaec527553 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:08.737974 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a966132d-9bc6-404a-af9e-d25803e1aa4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:08.738822 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a966132d-9bc6-404a-af9e-d25803e1aa4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:08.739113 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a966132d-9bc6-404a-af9e-d25803e1aa4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:08.739390 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a966132d-9bc6-404a-af9e-d25803e1aa4d admin admin] PATH_INFO: `/v3/projects/d004b77ebdd3487f96091b9e969b9c18` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:08.740576 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a966132d-9bc6-404a-af9e-d25803e1aa4d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d004b77ebdd3487f96091b9e969b9c18)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:08.742542 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a966132d-9bc6-404a-af9e-d25803e1aa4d admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:08.795083 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a966132d-9bc6-404a-af9e-d25803e1aa4d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd004b77ebdd3487f96091b9e969b9c18', 'request_id': 'req-a966132d-9bc6-404a-af9e-d25803e1aa4d'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:08.802119 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a966132d-9bc6-404a-af9e-d25803e1aa4d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd004b77ebdd3487f96091b9e969b9c18', 'request_id': 'req-a966132d-9bc6-404a-af9e-d25803e1aa4d'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:08.809602 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a966132d-9bc6-404a-af9e-d25803e1aa4d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd004b77ebdd3487f96091b9e969b9c18', 'request_id': 'req-a966132d-9bc6-404a-af9e-d25803e1aa4d'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:08.823340 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a966132d-9bc6-404a-af9e-d25803e1aa4d admin admin] The token cache is being invalidate because project d004b77ebdd3487f96091b9e969b9c18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:44:08.824071 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a966132d-9bc6-404a-af9e-d25803e1aa4d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:08.824453 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a966132d-9bc6-404a-af9e-d25803e1aa4d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:08.826367 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1313/2616] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Aug 6 01:44:08 2022] DELETE /identity/v3/projects/d004b77ebdd3487f96091b9e969b9c18 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:44:08.830476 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc356b7e-537b-48a5-8696-5fd0b89bf063 service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:08.840180 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc356b7e-537b-48a5-8696-5fd0b89bf063 service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:08.900690 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a84eca1-1517-4e65-bc0e-00b003c41674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:08.901396 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a84eca1-1517-4e65-bc0e-00b003c41674 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:08.901684 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a84eca1-1517-4e65-bc0e-00b003c41674 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:08.901969 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a84eca1-1517-4e65-bc0e-00b003c41674 admin admin] PATH_INFO: `/v3/users/509a9e84238948f68cc6a4181f7cda46` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:08.903226 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a84eca1-1517-4e65-bc0e-00b003c41674 admin admin] RBAC: Authorizing `identity:delete_user(user_id=509a9e84238948f68cc6a4181f7cda46)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:08.905271 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a84eca1-1517-4e65-bc0e-00b003c41674 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:08.927700 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a966132d-9bc6-404a-af9e-d25803e1aa4d admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:08.929503 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-a966132d-9bc6-404a-af9e-d25803e1aa4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:08.946418 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a966132d-9bc6-404a-af9e-d25803e1aa4d admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:08.953524 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861fbc61-d7dd-4576-8608-0d06fc66e086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:08.954318 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861fbc61-d7dd-4576-8608-0d06fc66e086 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:08.954570 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861fbc61-d7dd-4576-8608-0d06fc66e086 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:08.954766 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861fbc61-d7dd-4576-8608-0d06fc66e086 admin admin] PATH_INFO: `/v3/projects/0dff934cf5b7463b801c8b879eeb41cc` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:08.956102 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861fbc61-d7dd-4576-8608-0d06fc66e086 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0dff934cf5b7463b801c8b879eeb41cc)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:08.958291 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-9a84eca1-1517-4e65-bc0e-00b003c41674 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '509a9e84238948f68cc6a4181f7cda46'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:08.958621 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861fbc61-d7dd-4576-8608-0d06fc66e086 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:08.959218 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-9a84eca1-1517-4e65-bc0e-00b003c41674 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '509a9e84238948f68cc6a4181f7cda46'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:08.959839 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-9a84eca1-1517-4e65-bc0e-00b003c41674 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '509a9e84238948f68cc6a4181f7cda46'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:08.972603 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-9a84eca1-1517-4e65-bc0e-00b003c41674 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '509a9e84238948f68cc6a4181f7cda46'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:08.979478 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-9a84eca1-1517-4e65-bc0e-00b003c41674 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '509a9e84238948f68cc6a4181f7cda46'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:08.989504 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:44:08.996343 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1304/2617] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Aug 6 01:44:08 2022] DELETE /identity/v3/users/509a9e84238948f68cc6a4181f7cda46 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:44:09.016788 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-861fbc61-d7dd-4576-8608-0d06fc66e086 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0dff934cf5b7463b801c8b879eeb41cc', 'request_id': 'req-861fbc61-d7dd-4576-8608-0d06fc66e086'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:09.023198 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-861fbc61-d7dd-4576-8608-0d06fc66e086 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0dff934cf5b7463b801c8b879eeb41cc', 'request_id': 'req-861fbc61-d7dd-4576-8608-0d06fc66e086'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:09.030172 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-861fbc61-d7dd-4576-8608-0d06fc66e086 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0dff934cf5b7463b801c8b879eeb41cc', 'request_id': 'req-861fbc61-d7dd-4576-8608-0d06fc66e086'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:09.042809 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-861fbc61-d7dd-4576-8608-0d06fc66e086 admin admin] The token cache is being invalidate because project 0dff934cf5b7463b801c8b879eeb41cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:44:09.043129 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-861fbc61-d7dd-4576-8608-0d06fc66e086 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:09.043766 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-861fbc61-d7dd-4576-8608-0d06fc66e086 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:09.045676 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1314/2618] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Aug 6 01:44:08 2022] DELETE /identity/v3/projects/0dff934cf5b7463b801c8b879eeb41cc => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:44:09.049886 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a84eca1-1517-4e65-bc0e-00b003c41674 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:09.051628 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-9a84eca1-1517-4e65-bc0e-00b003c41674 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:09.121486 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a84eca1-1517-4e65-bc0e-00b003c41674 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:09.126826 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfed145-0071-4f1d-9220-163b4c9052fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:09.127507 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfed145-0071-4f1d-9220-163b4c9052fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:09.127794 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfed145-0071-4f1d-9220-163b4c9052fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:09.128071 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfed145-0071-4f1d-9220-163b4c9052fc admin admin] PATH_INFO: `/v3/users/f69d83c138984fbc9cdc574b99c9f773` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:09.129618 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfed145-0071-4f1d-9220-163b4c9052fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=f69d83c138984fbc9cdc574b99c9f773)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:09.131503 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfed145-0071-4f1d-9220-163b4c9052fc admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:09.185579 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-6bfed145-0071-4f1d-9220-163b4c9052fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f69d83c138984fbc9cdc574b99c9f773'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:09.186305 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-6bfed145-0071-4f1d-9220-163b4c9052fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f69d83c138984fbc9cdc574b99c9f773'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:09.186976 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-6bfed145-0071-4f1d-9220-163b4c9052fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f69d83c138984fbc9cdc574b99c9f773'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:09.202070 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-6bfed145-0071-4f1d-9220-163b4c9052fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f69d83c138984fbc9cdc574b99c9f773'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:09.209401 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-6bfed145-0071-4f1d-9220-163b4c9052fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f69d83c138984fbc9cdc574b99c9f773'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:09.223730 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1305/2619] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Aug 6 01:44:09 2022] DELETE /identity/v3/users/f69d83c138984fbc9cdc574b99c9f773 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:44:09.287121 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861fbc61-d7dd-4576-8608-0d06fc66e086 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:09.289081 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-861fbc61-d7dd-4576-8608-0d06fc66e086 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:09.342439 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:44:09.367470 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861fbc61-d7dd-4576-8608-0d06fc66e086 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:09.433523 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91a2c51-951d-4e78-a11f-3d72c276d1ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:09.434175 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91a2c51-951d-4e78-a11f-3d72c276d1ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:09.434461 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91a2c51-951d-4e78-a11f-3d72c276d1ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:09.434738 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91a2c51-951d-4e78-a11f-3d72c276d1ec admin admin] PATH_INFO: `/v3/projects/76f10ec322d04039817e415ff6d7b326` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:09.435949 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91a2c51-951d-4e78-a11f-3d72c276d1ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=76f10ec322d04039817e415ff6d7b326)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:09.437951 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91a2c51-951d-4e78-a11f-3d72c276d1ec admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:09.487855 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-f91a2c51-951d-4e78-a11f-3d72c276d1ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76f10ec322d04039817e415ff6d7b326', 'request_id': 'req-f91a2c51-951d-4e78-a11f-3d72c276d1ec'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:09.497475 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-f91a2c51-951d-4e78-a11f-3d72c276d1ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76f10ec322d04039817e415ff6d7b326', 'request_id': 'req-f91a2c51-951d-4e78-a11f-3d72c276d1ec'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:09.504525 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-f91a2c51-951d-4e78-a11f-3d72c276d1ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76f10ec322d04039817e415ff6d7b326', 'request_id': 'req-f91a2c51-951d-4e78-a11f-3d72c276d1ec'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:09.517519 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfed145-0071-4f1d-9220-163b4c9052fc admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:09.519057 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-6bfed145-0071-4f1d-9220-163b4c9052fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:09.520124 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-f91a2c51-951d-4e78-a11f-3d72c276d1ec admin admin] The token cache is being invalidate because project 76f10ec322d04039817e415ff6d7b326 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:44:09.520478 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-f91a2c51-951d-4e78-a11f-3d72c276d1ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:09.521211 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-f91a2c51-951d-4e78-a11f-3d72c276d1ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:09.523028 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1315/2620] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Aug 6 01:44:09 2022] DELETE /identity/v3/projects/76f10ec322d04039817e415ff6d7b326 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:44:09.527039 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91a2c51-951d-4e78-a11f-3d72c276d1ec admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:09.528419 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-f91a2c51-951d-4e78-a11f-3d72c276d1ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:09.551152 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfed145-0071-4f1d-9220-163b4c9052fc admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:09.597934 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91a2c51-951d-4e78-a11f-3d72c276d1ec admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:09.602519 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bdbb28-a97a-45d7-8173-bc40703d1c37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:09.603155 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bdbb28-a97a-45d7-8173-bc40703d1c37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:09.603431 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bdbb28-a97a-45d7-8173-bc40703d1c37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:09.603703 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bdbb28-a97a-45d7-8173-bc40703d1c37 admin admin] PATH_INFO: `/v3/users/9635fc61a5234788906dda89c67bba85` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:09.604871 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bdbb28-a97a-45d7-8173-bc40703d1c37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9635fc61a5234788906dda89c67bba85)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:09.607365 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bdbb28-a97a-45d7-8173-bc40703d1c37 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:09.611286 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb2493e-ec64-488d-816f-43745a1c72bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:09.612005 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb2493e-ec64-488d-816f-43745a1c72bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:09.612355 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb2493e-ec64-488d-816f-43745a1c72bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:09.612699 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb2493e-ec64-488d-816f-43745a1c72bf admin admin] PATH_INFO: `/v3/projects/7cc4b0c35764426284320c4922333ae7` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:09.614060 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb2493e-ec64-488d-816f-43745a1c72bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=7cc4b0c35764426284320c4922333ae7)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:09.615945 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb2493e-ec64-488d-816f-43745a1c72bf admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:09.629554 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:44:09.643944 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:44:09.662382 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-99bdbb28-a97a-45d7-8173-bc40703d1c37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9635fc61a5234788906dda89c67bba85'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:09.670797 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-99bdbb28-a97a-45d7-8173-bc40703d1c37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9635fc61a5234788906dda89c67bba85'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:09.671258 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-99bdbb28-a97a-45d7-8173-bc40703d1c37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9635fc61a5234788906dda89c67bba85'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:09.671879 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-99bdbb28-a97a-45d7-8173-bc40703d1c37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9635fc61a5234788906dda89c67bba85'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:09.673750 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-5fb2493e-ec64-488d-816f-43745a1c72bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7cc4b0c35764426284320c4922333ae7', 'request_id': 'req-5fb2493e-ec64-488d-816f-43745a1c72bf'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:09.680201 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-5fb2493e-ec64-488d-816f-43745a1c72bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7cc4b0c35764426284320c4922333ae7', 'request_id': 'req-5fb2493e-ec64-488d-816f-43745a1c72bf'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:09.684189 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-99bdbb28-a97a-45d7-8173-bc40703d1c37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9635fc61a5234788906dda89c67bba85'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:09.686459 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-5fb2493e-ec64-488d-816f-43745a1c72bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7cc4b0c35764426284320c4922333ae7', 'request_id': 'req-5fb2493e-ec64-488d-816f-43745a1c72bf'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:09.697632 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1316/2621] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Aug 6 01:44:09 2022] DELETE /identity/v3/users/9635fc61a5234788906dda89c67bba85 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:44:09.699119 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-5fb2493e-ec64-488d-816f-43745a1c72bf admin admin] The token cache is being invalidate because project 7cc4b0c35764426284320c4922333ae7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:44:09.699407 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-5fb2493e-ec64-488d-816f-43745a1c72bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:09.700051 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-5fb2493e-ec64-488d-816f-43745a1c72bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:09.702252 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1306/2622] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Aug 6 01:44:09 2022] DELETE /identity/v3/projects/7cc4b0c35764426284320c4922333ae7 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:44:09.706911 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7274540c-0aa1-4eea-8380-80ee9623eebb None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:09.707274 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7274540c-0aa1-4eea-8380-80ee9623eebb None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:09.707623 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7274540c-0aa1-4eea-8380-80ee9623eebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:09.728490 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-7274540c-0aa1-4eea-8380-80ee9623eebb None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:44:09.729757 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-7274540c-0aa1-4eea-8380-80ee9623eebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:09.795932 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1317/2623] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:44:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:44:09.805349 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e1fa08-c79d-4dbb-ae77-55e067ccf47b None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:09.805709 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e1fa08-c79d-4dbb-ae77-55e067ccf47b None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:09.806057 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e1fa08-c79d-4dbb-ae77-55e067ccf47b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:09.827971 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-c8e1fa08-c79d-4dbb-ae77-55e067ccf47b None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:44:09.829231 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-c8e1fa08-c79d-4dbb-ae77-55e067ccf47b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:09.838161 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1307/2624] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:44:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:44:09.843223 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7274540c-0aa1-4eea-8380-80ee9623eebb None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:09.853584 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7274540c-0aa1-4eea-8380-80ee9623eebb None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:09.859006 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd258e2-3e14-4ee2-96c7-128620e1a8e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:09.859811 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd258e2-3e14-4ee2-96c7-128620e1a8e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:09.860269 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd258e2-3e14-4ee2-96c7-128620e1a8e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:09.860699 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd258e2-3e14-4ee2-96c7-128620e1a8e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:09.861596 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd258e2-3e14-4ee2-96c7-128620e1a8e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:44:09.862022 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd258e2-3e14-4ee2-96c7-128620e1a8e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:09.863498 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd258e2-3e14-4ee2-96c7-128620e1a8e6 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:09.879803 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1318/2625] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Aug 6 01:44:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:44:09.881227 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e1fa08-c79d-4dbb-ae77-55e067ccf47b None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:09.891107 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e1fa08-c79d-4dbb-ae77-55e067ccf47b None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:09.895845 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7341c3-a681-4e0e-b985-d6f56440ce56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:09.896445 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7341c3-a681-4e0e-b985-d6f56440ce56 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:09.896732 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7341c3-a681-4e0e-b985-d6f56440ce56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:09.897014 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7341c3-a681-4e0e-b985-d6f56440ce56 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:09.897724 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7341c3-a681-4e0e-b985-d6f56440ce56 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:09.899628 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7341c3-a681-4e0e-b985-d6f56440ce56 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:09.927013 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1308/2626] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Aug 6 01:44:09 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:44:09.931157 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd258e2-3e14-4ee2-96c7-128620e1a8e6 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:09.943732 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd258e2-3e14-4ee2-96c7-128620e1a8e6 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:09.998186 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7341c3-a681-4e0e-b985-d6f56440ce56 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:10.000140 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-4a7341c3-a681-4e0e-b985-d6f56440ce56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:10.014728 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b741d4c7-4f94-43a6-86fa-5b970bd31e91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:10.015590 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b741d4c7-4f94-43a6-86fa-5b970bd31e91 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:10.015898 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b741d4c7-4f94-43a6-86fa-5b970bd31e91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:10.016275 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b741d4c7-4f94-43a6-86fa-5b970bd31e91 admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:10.017248 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b741d4c7-4f94-43a6-86fa-5b970bd31e91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:10.019544 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b741d4c7-4f94-43a6-86fa-5b970bd31e91 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:10.045622 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1319/2627] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:44:09 2022] POST /identity/v3/users => generated 293 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:44:10.049714 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b741d4c7-4f94-43a6-86fa-5b970bd31e91 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:10.059467 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b741d4c7-4f94-43a6-86fa-5b970bd31e91 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:10.065037 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc99e94-25d1-43b1-8f91-6822a95256f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:10.065809 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc99e94-25d1-43b1-8f91-6822a95256f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:10.066094 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc99e94-25d1-43b1-8f91-6822a95256f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:10.066371 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc99e94-25d1-43b1-8f91-6822a95256f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:10.067072 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc99e94-25d1-43b1-8f91-6822a95256f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:10.068668 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc99e94-25d1-43b1-8f91-6822a95256f7 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:10.073694 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7341c3-a681-4e0e-b985-d6f56440ce56 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:10.078608 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4726ea6-8e56-4f86-be16-2ecb64fb341e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:10.079249 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4726ea6-8e56-4f86-be16-2ecb64fb341e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:10.079534 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4726ea6-8e56-4f86-be16-2ecb64fb341e admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:10.079643 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1320/2628] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:44:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 01:44:10.079974 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4726ea6-8e56-4f86-be16-2ecb64fb341e admin admin] PATH_INFO: `/v3/projects/ffb5f1b7010642d0ad57752aecde8756` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:10.081263 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4726ea6-8e56-4f86-be16-2ecb64fb341e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffb5f1b7010642d0ad57752aecde8756)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:10.083278 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4726ea6-8e56-4f86-be16-2ecb64fb341e admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:10.083419 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc99e94-25d1-43b1-8f91-6822a95256f7 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:10.088331 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:44:10.094487 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc99e94-25d1-43b1-8f91-6822a95256f7 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:10.101105 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186d0068-d4c4-4b6c-964c-921482ed6c03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:10.101863 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186d0068-d4c4-4b6c-964c-921482ed6c03 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:10.102146 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186d0068-d4c4-4b6c-964c-921482ed6c03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:10.102424 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186d0068-d4c4-4b6c-964c-921482ed6c03 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:10.103142 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186d0068-d4c4-4b6c-964c-921482ed6c03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:10.104732 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186d0068-d4c4-4b6c-964c-921482ed6c03 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:10.115986 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1321/2629] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:44:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 01:44:10.120205 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186d0068-d4c4-4b6c-964c-921482ed6c03 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:10.133178 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186d0068-d4c4-4b6c-964c-921482ed6c03 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:10.137126 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-c4726ea6-8e56-4f86-be16-2ecb64fb341e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffb5f1b7010642d0ad57752aecde8756', 'request_id': 'req-c4726ea6-8e56-4f86-be16-2ecb64fb341e'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:10.144009 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-c4726ea6-8e56-4f86-be16-2ecb64fb341e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffb5f1b7010642d0ad57752aecde8756', 'request_id': 'req-c4726ea6-8e56-4f86-be16-2ecb64fb341e'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:10.151153 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-c4726ea6-8e56-4f86-be16-2ecb64fb341e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffb5f1b7010642d0ad57752aecde8756', 'request_id': 'req-c4726ea6-8e56-4f86-be16-2ecb64fb341e'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:10.167218 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-c4726ea6-8e56-4f86-be16-2ecb64fb341e admin admin] The token cache is being invalidate because project ffb5f1b7010642d0ad57752aecde8756 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:44:10.167562 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-c4726ea6-8e56-4f86-be16-2ecb64fb341e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:10.168188 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-c4726ea6-8e56-4f86-be16-2ecb64fb341e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:10.170450 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1309/2630] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Aug 6 01:44:09 2022] DELETE /identity/v3/projects/ffb5f1b7010642d0ad57752aecde8756 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:44:10.176391 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb89ede7-64bd-4112-b5c5-385a78240fad None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:10.176766 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb89ede7-64bd-4112-b5c5-385a78240fad None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:10.177149 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb89ede7-64bd-4112-b5c5-385a78240fad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:10.202132 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-eb89ede7-64bd-4112-b5c5-385a78240fad None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:44:10.202132 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-eb89ede7-64bd-4112-b5c5-385a78240fad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:10.206575 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:44:10.211858 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56b292a-5758-41b8-96ec-bb35f38a5efa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:10.212598 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56b292a-5758-41b8-96ec-bb35f38a5efa admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:10.212886 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56b292a-5758-41b8-96ec-bb35f38a5efa admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:10.213197 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56b292a-5758-41b8-96ec-bb35f38a5efa admin admin] PATH_INFO: `/v3/projects/947dd3c9188d48d7a304d3150659f53a/users/90a4ea1f21e84e5a912055ad7cc80bdd/roles/61b3b93d37a047e4bb8369b46fe20db9` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:10.230730 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56b292a-5758-41b8-96ec-bb35f38a5efa admin admin] RBAC: Authorizing `identity:create_grant(project_id=947dd3c9188d48d7a304d3150659f53a, user_id=90a4ea1f21e84e5a912055ad7cc80bdd, role_id=61b3b93d37a047e4bb8369b46fe20db9)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:10.234283 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56b292a-5758-41b8-96ec-bb35f38a5efa admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:10.250604 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1322/2631] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:44:10 2022] PUT /identity/v3/projects/947dd3c9188d48d7a304d3150659f53a/users/90a4ea1f21e84e5a912055ad7cc80bdd/roles/61b3b93d37a047e4bb8369b46fe20db9 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:44:10.261308 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f39ed8-f576-4eef-9d79-1b8541ba85a0 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:10.261684 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f39ed8-f576-4eef-9d79-1b8541ba85a0 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:10.262031 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f39ed8-f576-4eef-9d79-1b8541ba85a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:10.294218 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-97f39ed8-f576-4eef-9d79-1b8541ba85a0 None None] MFA Rules not processed for user `90a4ea1f21e84e5a912055ad7cc80bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:44:10.295209 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1310/2632] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:44:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:44:10.297171 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-97f39ed8-f576-4eef-9d79-1b8541ba85a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:10.304971 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227b9af0-d350-4083-882f-bb7705426150 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:10.305303 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227b9af0-d350-4083-882f-bb7705426150 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:10.305588 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227b9af0-d350-4083-882f-bb7705426150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:10.327323 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-227b9af0-d350-4083-882f-bb7705426150 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:44:10.328697 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-227b9af0-d350-4083-882f-bb7705426150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:10.385089 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1323/2633] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:44:10 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:44:10.396562 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f39ed8-f576-4eef-9d79-1b8541ba85a0 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:10.400856 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-97f39ed8-f576-4eef-9d79-1b8541ba85a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:10.407084 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1311/2634] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:44:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:44:10.412798 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227b9af0-d350-4083-882f-bb7705426150 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:10.422955 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227b9af0-d350-4083-882f-bb7705426150 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:10.423508 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f39ed8-f576-4eef-9d79-1b8541ba85a0 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:10.428255 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c6a0a7-b625-49bc-8de1-01add0098cd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:10.428685 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f3ffb6-2e1e-476d-b258-11e04a820c7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:10.428890 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c6a0a7-b625-49bc-8de1-01add0098cd6 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:10.429170 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c6a0a7-b625-49bc-8de1-01add0098cd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:10.429296 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f3ffb6-2e1e-476d-b258-11e04a820c7d admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:10.429445 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c6a0a7-b625-49bc-8de1-01add0098cd6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:10.429557 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f3ffb6-2e1e-476d-b258-11e04a820c7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:10.429788 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f3ffb6-2e1e-476d-b258-11e04a820c7d admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:10.430138 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c6a0a7-b625-49bc-8de1-01add0098cd6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:44:10.430449 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c6a0a7-b625-49bc-8de1-01add0098cd6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:10.430584 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f3ffb6-2e1e-476d-b258-11e04a820c7d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:10.432256 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c6a0a7-b625-49bc-8de1-01add0098cd6 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:10.432929 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f3ffb6-2e1e-476d-b258-11e04a820c7d admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:10.449938 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1312/2635] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Aug 6 01:44:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:44:10.451978 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c6a0a7-b625-49bc-8de1-01add0098cd6 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:10.462692 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c6a0a7-b625-49bc-8de1-01add0098cd6 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:10.466642 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1324/2636] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Aug 6 01:44:10 2022] POST /identity/v3/projects => generated 385 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:44:10.469211 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc51d855-c9a5-46ba-99cb-734ca10650e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:10.469912 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc51d855-c9a5-46ba-99cb-734ca10650e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:10.470207 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc51d855-c9a5-46ba-99cb-734ca10650e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:10.470488 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc51d855-c9a5-46ba-99cb-734ca10650e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:10.470746 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f3ffb6-2e1e-476d-b258-11e04a820c7d admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:10.471163 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc51d855-c9a5-46ba-99cb-734ca10650e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:10.473097 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc51d855-c9a5-46ba-99cb-734ca10650e0 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:10.506019 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1313/2637] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Aug 6 01:44:10 2022] POST /identity/v3/projects => generated 387 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:44:10.510260 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc51d855-c9a5-46ba-99cb-734ca10650e0 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:10.521302 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:44:10.544481 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f3ffb6-2e1e-476d-b258-11e04a820c7d admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:10.586856 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc51d855-c9a5-46ba-99cb-734ca10650e0 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:10.592457 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6faaf8b-6d0c-4c81-a2b9-a250a4474002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:10.593264 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6faaf8b-6d0c-4c81-a2b9-a250a4474002 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:10.593549 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6faaf8b-6d0c-4c81-a2b9-a250a4474002 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:10.593888 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6faaf8b-6d0c-4c81-a2b9-a250a4474002 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:10.594570 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6faaf8b-6d0c-4c81-a2b9-a250a4474002 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:10.596494 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6faaf8b-6d0c-4c81-a2b9-a250a4474002 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:10.603555 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208ac765-9b53-494e-9cd2-47d9ca3e546f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:10.604187 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208ac765-9b53-494e-9cd2-47d9ca3e546f admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:10.604475 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208ac765-9b53-494e-9cd2-47d9ca3e546f admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:10.604752 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208ac765-9b53-494e-9cd2-47d9ca3e546f admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:10.605463 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208ac765-9b53-494e-9cd2-47d9ca3e546f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:10.607974 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208ac765-9b53-494e-9cd2-47d9ca3e546f admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:10.620164 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1314/2638] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:44:10 2022] POST /identity/v3/users => generated 294 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:44:10.626432 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6faaf8b-6d0c-4c81-a2b9-a250a4474002 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:10.636531 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1325/2639] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:44:10 2022] POST /identity/v3/users => generated 293 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:44:10.637590 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208ac765-9b53-494e-9cd2-47d9ca3e546f admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:10.638927 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6faaf8b-6d0c-4c81-a2b9-a250a4474002 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:10.644720 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002d82fc-9f68-497f-9680-0d896358b3b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:10.645371 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002d82fc-9f68-497f-9680-0d896358b3b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:10.645660 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002d82fc-9f68-497f-9680-0d896358b3b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:10.645935 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002d82fc-9f68-497f-9680-0d896358b3b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:10.646637 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002d82fc-9f68-497f-9680-0d896358b3b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:10.648184 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208ac765-9b53-494e-9cd2-47d9ca3e546f admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:10.648318 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002d82fc-9f68-497f-9680-0d896358b3b4 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:10.649978 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:44:10.653539 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6879a8-9990-4d96-8a49-4de9ec65a460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:10.654313 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6879a8-9990-4d96-8a49-4de9ec65a460 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:10.654803 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6879a8-9990-4d96-8a49-4de9ec65a460 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:10.655087 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6879a8-9990-4d96-8a49-4de9ec65a460 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:10.655826 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6879a8-9990-4d96-8a49-4de9ec65a460 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:10.657578 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6879a8-9990-4d96-8a49-4de9ec65a460 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:10.662397 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1315/2640] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:44:10 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 01:44:10.666563 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002d82fc-9f68-497f-9680-0d896358b3b4 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:10.668033 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1326/2641] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:44:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:44:10.672521 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6879a8-9990-4d96-8a49-4de9ec65a460 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:10.680577 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002d82fc-9f68-497f-9680-0d896358b3b4 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:10.682643 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6879a8-9990-4d96-8a49-4de9ec65a460 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:10.686758 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54077e3-2bba-42c8-a51e-d059b331acbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:10.687980 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54077e3-2bba-42c8-a51e-d059b331acbf admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:10.688112 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92970bb-ebd6-4fc5-8ee6-5dfc48be3bc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:10.688258 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54077e3-2bba-42c8-a51e-d059b331acbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:10.688541 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54077e3-2bba-42c8-a51e-d059b331acbf admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:10.688654 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92970bb-ebd6-4fc5-8ee6-5dfc48be3bc7 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:10.688909 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92970bb-ebd6-4fc5-8ee6-5dfc48be3bc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:10.689227 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92970bb-ebd6-4fc5-8ee6-5dfc48be3bc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:10.689346 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54077e3-2bba-42c8-a51e-d059b331acbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:10.689919 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92970bb-ebd6-4fc5-8ee6-5dfc48be3bc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:10.690904 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54077e3-2bba-42c8-a51e-d059b331acbf admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:10.691494 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92970bb-ebd6-4fc5-8ee6-5dfc48be3bc7 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:10.702471 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1316/2642] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:44:10 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 01:44:10.703646 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1327/2643] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:44:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 01:44:10.706847 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54077e3-2bba-42c8-a51e-d059b331acbf admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:10.707002 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92970bb-ebd6-4fc5-8ee6-5dfc48be3bc7 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:10.716998 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92970bb-ebd6-4fc5-8ee6-5dfc48be3bc7 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:10.717875 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54077e3-2bba-42c8-a51e-d059b331acbf admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:10.721899 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae21e3c7-0b6f-42d3-b9ca-261078098575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:10.722576 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae21e3c7-0b6f-42d3-b9ca-261078098575 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:10.722860 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae21e3c7-0b6f-42d3-b9ca-261078098575 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:10.723144 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae21e3c7-0b6f-42d3-b9ca-261078098575 admin admin] PATH_INFO: `/v3/projects/38dc5750595b4a43a66581fa56689bd3/users/97776fedf80b4d07834007e5686978ce/roles/61b3b93d37a047e4bb8369b46fe20db9` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:10.725261 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42b42d2-7b32-4446-95b0-2be2b70218ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:10.726403 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42b42d2-7b32-4446-95b0-2be2b70218ef admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:10.726890 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42b42d2-7b32-4446-95b0-2be2b70218ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:10.727370 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42b42d2-7b32-4446-95b0-2be2b70218ef admin admin] PATH_INFO: `/v3/projects/a5e8bc0340ca4134ac5f55e003a72545/users/8c14d03bf3164d71886a70c24772f503/roles/61b3b93d37a047e4bb8369b46fe20db9` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:10.739982 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae21e3c7-0b6f-42d3-b9ca-261078098575 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38dc5750595b4a43a66581fa56689bd3, user_id=97776fedf80b4d07834007e5686978ce, role_id=61b3b93d37a047e4bb8369b46fe20db9)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:10.742734 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae21e3c7-0b6f-42d3-b9ca-261078098575 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:10.744293 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42b42d2-7b32-4446-95b0-2be2b70218ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5e8bc0340ca4134ac5f55e003a72545, user_id=8c14d03bf3164d71886a70c24772f503, role_id=61b3b93d37a047e4bb8369b46fe20db9)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:10.747341 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42b42d2-7b32-4446-95b0-2be2b70218ef admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:10.758238 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1328/2644] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:44:10 2022] PUT /identity/v3/projects/38dc5750595b4a43a66581fa56689bd3/users/97776fedf80b4d07834007e5686978ce/roles/61b3b93d37a047e4bb8369b46fe20db9 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:44:10.762216 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1317/2645] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:44:10 2022] PUT /identity/v3/projects/a5e8bc0340ca4134ac5f55e003a72545/users/8c14d03bf3164d71886a70c24772f503/roles/61b3b93d37a047e4bb8369b46fe20db9 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:44:10.771488 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cb457d-0e86-40d7-be61-eedabb52178f None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:10.771803 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cb457d-0e86-40d7-be61-eedabb52178f None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:10.772081 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cb457d-0e86-40d7-be61-eedabb52178f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:10.804779 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-08cb457d-0e86-40d7-be61-eedabb52178f None None] MFA Rules not processed for user `97776fedf80b4d07834007e5686978ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:44:10.806718 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-08cb457d-0e86-40d7-be61-eedabb52178f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:10.838905 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d20edb9-51a5-4044-b4d3-0775f13f622f None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:10.839118 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d20edb9-51a5-4044-b4d3-0775f13f622f None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:10.839390 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d20edb9-51a5-4044-b4d3-0775f13f622f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:10.870463 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-9d20edb9-51a5-4044-b4d3-0775f13f622f None None] MFA Rules not processed for user `8c14d03bf3164d71886a70c24772f503`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:44:10.871981 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-9d20edb9-51a5-4044-b4d3-0775f13f622f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:10.882393 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1329/2646] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:44:10 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:44:10.886984 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cb457d-0e86-40d7-be61-eedabb52178f None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:10.913158 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:44:10.947632 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1318/2647] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:44:10 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:44:10.952368 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d20edb9-51a5-4044-b4d3-0775f13f622f None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:10.954077 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cb457d-0e86-40d7-be61-eedabb52178f None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:10.958802 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875501dd-98f3-4e4f-b266-336504a79ab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:10.959418 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875501dd-98f3-4e4f-b266-336504a79ab0 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:10.959695 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875501dd-98f3-4e4f-b266-336504a79ab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:10.959968 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875501dd-98f3-4e4f-b266-336504a79ab0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:10.960666 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875501dd-98f3-4e4f-b266-336504a79ab0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:10.962657 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875501dd-98f3-4e4f-b266-336504a79ab0 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:10.962920 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d20edb9-51a5-4044-b4d3-0775f13f622f None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:10.968086 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c986d5-6bb8-40f7-a68f-0ea66e61bede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:10.968694 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c986d5-6bb8-40f7-a68f-0ea66e61bede admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:10.968978 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c986d5-6bb8-40f7-a68f-0ea66e61bede admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:10.969280 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c986d5-6bb8-40f7-a68f-0ea66e61bede admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:10.969936 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c986d5-6bb8-40f7-a68f-0ea66e61bede admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:10.971810 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c986d5-6bb8-40f7-a68f-0ea66e61bede admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:10.990907 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1330/2648] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Aug 6 01:44:10 2022] POST /identity/v3/projects => generated 387 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:44:10.995090 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875501dd-98f3-4e4f-b266-336504a79ab0 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:11.000367 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1319/2649] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Aug 6 01:44:10 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:44:11.004425 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c986d5-6bb8-40f7-a68f-0ea66e61bede admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:11.058074 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875501dd-98f3-4e4f-b266-336504a79ab0 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:11.068225 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c986d5-6bb8-40f7-a68f-0ea66e61bede admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:11.072985 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38325fc0-ad20-4f67-9295-19799dc1fb66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:11.073629 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38325fc0-ad20-4f67-9295-19799dc1fb66 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:11.073914 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38325fc0-ad20-4f67-9295-19799dc1fb66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:11.074194 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38325fc0-ad20-4f67-9295-19799dc1fb66 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:11.074854 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38325fc0-ad20-4f67-9295-19799dc1fb66 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:11.077450 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38325fc0-ad20-4f67-9295-19799dc1fb66 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:11.098662 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1320/2650] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:44:11 2022] POST /identity/v3/users => generated 299 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:44:11.102612 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38325fc0-ad20-4f67-9295-19799dc1fb66 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:11.112352 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38325fc0-ad20-4f67-9295-19799dc1fb66 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:11.117001 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb5c97f-5d9a-4c62-8fbc-8be182a24c9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:11.117394 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70cb803-60bb-4605-a5ee-76ecc3cac148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:11.117663 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb5c97f-5d9a-4c62-8fbc-8be182a24c9a admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:11.117947 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb5c97f-5d9a-4c62-8fbc-8be182a24c9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:11.118054 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70cb803-60bb-4605-a5ee-76ecc3cac148 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:11.118223 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb5c97f-5d9a-4c62-8fbc-8be182a24c9a admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:11.118337 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70cb803-60bb-4605-a5ee-76ecc3cac148 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:11.118538 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70cb803-60bb-4605-a5ee-76ecc3cac148 admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:11.118909 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb5c97f-5d9a-4c62-8fbc-8be182a24c9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:11.119209 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70cb803-60bb-4605-a5ee-76ecc3cac148 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:11.120810 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb5c97f-5d9a-4c62-8fbc-8be182a24c9a admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:11.121899 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70cb803-60bb-4605-a5ee-76ecc3cac148 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:11.132901 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1321/2651] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:44:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 01:44:11.137189 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb5c97f-5d9a-4c62-8fbc-8be182a24c9a admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:11.148284 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1331/2652] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:44:10 2022] POST /identity/v3/users => generated 300 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:44:11.148483 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb5c97f-5d9a-4c62-8fbc-8be182a24c9a admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:11.149430 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70cb803-60bb-4605-a5ee-76ecc3cac148 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:11.152275 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d86adac-2dc6-413e-9c59-85ea476d89e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:11.152932 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d86adac-2dc6-413e-9c59-85ea476d89e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:11.153249 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d86adac-2dc6-413e-9c59-85ea476d89e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:11.153529 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d86adac-2dc6-413e-9c59-85ea476d89e3 admin admin] PATH_INFO: `/v3/projects/6c53ce57303846acb7d840931f72b4f0/users/694c80a63b0c48d2956c8a379d24dd59/roles/4559b9a0837249f895a13c70096f8d38` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:11.155946 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:44:11.161114 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70cb803-60bb-4605-a5ee-76ecc3cac148 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:11.166496 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a446b6c-0e52-492c-a6b8-978ebd5caddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:11.167123 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a446b6c-0e52-492c-a6b8-978ebd5caddc admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:11.167407 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a446b6c-0e52-492c-a6b8-978ebd5caddc admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:11.167684 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a446b6c-0e52-492c-a6b8-978ebd5caddc admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:11.168386 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a446b6c-0e52-492c-a6b8-978ebd5caddc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:11.170245 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a446b6c-0e52-492c-a6b8-978ebd5caddc admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:11.171034 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d86adac-2dc6-413e-9c59-85ea476d89e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c53ce57303846acb7d840931f72b4f0, user_id=694c80a63b0c48d2956c8a379d24dd59, role_id=4559b9a0837249f895a13c70096f8d38)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:11.173750 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d86adac-2dc6-413e-9c59-85ea476d89e3 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:11.180607 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1332/2653] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:44:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:44:11.184741 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a446b6c-0e52-492c-a6b8-978ebd5caddc admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:11.187981 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1322/2654] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:44:11 2022] PUT /identity/v3/projects/6c53ce57303846acb7d840931f72b4f0/users/694c80a63b0c48d2956c8a379d24dd59/roles/4559b9a0837249f895a13c70096f8d38 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:44:11.192159 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d86adac-2dc6-413e-9c59-85ea476d89e3 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:11.196358 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a446b6c-0e52-492c-a6b8-978ebd5caddc admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:11.226748 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:44:11.257800 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a659f0de-ec98-44ff-902e-1e4d45e3c0f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:11.258063 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d86adac-2dc6-413e-9c59-85ea476d89e3 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:11.258998 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a659f0de-ec98-44ff-902e-1e4d45e3c0f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:11.259515 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a659f0de-ec98-44ff-902e-1e4d45e3c0f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:11.260032 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a659f0de-ec98-44ff-902e-1e4d45e3c0f5 admin admin] PATH_INFO: `/v3/projects/244f8d9d29fd418abe5fea70dacca544/users/0118f08c0449477eaea77301906818f0/roles/4559b9a0837249f895a13c70096f8d38` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:11.263164 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277f71cb-88b8-41a8-97f8-586ef6db9fa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:11.263776 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277f71cb-88b8-41a8-97f8-586ef6db9fa4 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:11.264064 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277f71cb-88b8-41a8-97f8-586ef6db9fa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:11.264342 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277f71cb-88b8-41a8-97f8-586ef6db9fa4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:11.265056 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277f71cb-88b8-41a8-97f8-586ef6db9fa4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:11.266774 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277f71cb-88b8-41a8-97f8-586ef6db9fa4 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:11.280238 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1323/2655] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:44:11 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 01:44:11.280538 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a659f0de-ec98-44ff-902e-1e4d45e3c0f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=244f8d9d29fd418abe5fea70dacca544, user_id=0118f08c0449477eaea77301906818f0, role_id=4559b9a0837249f895a13c70096f8d38)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:11.281251 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277f71cb-88b8-41a8-97f8-586ef6db9fa4 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:11.283281 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a659f0de-ec98-44ff-902e-1e4d45e3c0f5 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:11.291308 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277f71cb-88b8-41a8-97f8-586ef6db9fa4 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:11.297139 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ce9af2-4e79-4374-949d-8064ced42a2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:11.297934 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ce9af2-4e79-4374-949d-8064ced42a2d admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:11.298478 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ce9af2-4e79-4374-949d-8064ced42a2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:11.298478 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ce9af2-4e79-4374-949d-8064ced42a2d admin admin] PATH_INFO: `/v3/projects/6c53ce57303846acb7d840931f72b4f0/users/694c80a63b0c48d2956c8a379d24dd59/roles/4559b9a0837249f895a13c70096f8d38` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:11.300447 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ce9af2-4e79-4374-949d-8064ced42a2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c53ce57303846acb7d840931f72b4f0, user_id=694c80a63b0c48d2956c8a379d24dd59, role_id=4559b9a0837249f895a13c70096f8d38)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:11.302738 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1333/2656] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:44:11 2022] PUT /identity/v3/projects/244f8d9d29fd418abe5fea70dacca544/users/0118f08c0449477eaea77301906818f0/roles/4559b9a0837249f895a13c70096f8d38 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:44:11.303114 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ce9af2-4e79-4374-949d-8064ced42a2d admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:11.306572 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a659f0de-ec98-44ff-902e-1e4d45e3c0f5 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:11.314743 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1324/2657] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:44:11 2022] PUT /identity/v3/projects/6c53ce57303846acb7d840931f72b4f0/users/694c80a63b0c48d2956c8a379d24dd59/roles/4559b9a0837249f895a13c70096f8d38 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:44:11.324822 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c817d2-1e4e-49b9-aafd-57f3d8d38713 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:11.325145 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c817d2-1e4e-49b9-aafd-57f3d8d38713 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:11.325452 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c817d2-1e4e-49b9-aafd-57f3d8d38713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:11.353236 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-23c817d2-1e4e-49b9-aafd-57f3d8d38713 None None] MFA Rules not processed for user `694c80a63b0c48d2956c8a379d24dd59`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:44:11.354547 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-23c817d2-1e4e-49b9-aafd-57f3d8d38713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:11.370761 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a659f0de-ec98-44ff-902e-1e4d45e3c0f5 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:11.429733 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5692849-add2-400b-b332-613db35108c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:11.430457 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5692849-add2-400b-b332-613db35108c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:11.430823 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5692849-add2-400b-b332-613db35108c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:11.431102 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5692849-add2-400b-b332-613db35108c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:11.431802 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5692849-add2-400b-b332-613db35108c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:11.433440 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5692849-add2-400b-b332-613db35108c5 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:11.434209 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1325/2658] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:44:11 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:44:11.444427 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1334/2659] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:44:11 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:44:11.446106 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c817d2-1e4e-49b9-aafd-57f3d8d38713 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:11.447792 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-23c817d2-1e4e-49b9-aafd-57f3d8d38713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:11.448227 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5692849-add2-400b-b332-613db35108c5 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:11.458820 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5692849-add2-400b-b332-613db35108c5 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:11.463479 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea24ddb5-ffbb-448d-b458-d765fa2df83a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:11.464131 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea24ddb5-ffbb-448d-b458-d765fa2df83a admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:11.464421 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea24ddb5-ffbb-448d-b458-d765fa2df83a admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:11.464704 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea24ddb5-ffbb-448d-b458-d765fa2df83a admin admin] PATH_INFO: `/v3/projects/244f8d9d29fd418abe5fea70dacca544/users/0118f08c0449477eaea77301906818f0/roles/4559b9a0837249f895a13c70096f8d38` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:11.467512 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea24ddb5-ffbb-448d-b458-d765fa2df83a admin admin] RBAC: Authorizing `identity:create_grant(project_id=244f8d9d29fd418abe5fea70dacca544, user_id=0118f08c0449477eaea77301906818f0, role_id=4559b9a0837249f895a13c70096f8d38)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:11.470349 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea24ddb5-ffbb-448d-b458-d765fa2df83a admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:11.481127 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1335/2660] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:44:11 2022] PUT /identity/v3/projects/244f8d9d29fd418abe5fea70dacca544/users/0118f08c0449477eaea77301906818f0/roles/4559b9a0837249f895a13c70096f8d38 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:44:11.491224 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff4f172-7436-44b0-a987-17c62b0ac944 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:11.491545 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff4f172-7436-44b0-a987-17c62b0ac944 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:11.491826 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff4f172-7436-44b0-a987-17c62b0ac944 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:11.505811 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c817d2-1e4e-49b9-aafd-57f3d8d38713 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:11.509383 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:44:11.520532 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-2ff4f172-7436-44b0-a987-17c62b0ac944 None None] MFA Rules not processed for user `0118f08c0449477eaea77301906818f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:44:11.521928 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-2ff4f172-7436-44b0-a987-17c62b0ac944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:11.558835 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:44:11.559666 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308ce6a9-d6ba-4a0a-92fd-33acc7224398 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1be1e6173e374a60b9e1fbf51a1ab571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:11.560720 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308ce6a9-d6ba-4a0a-92fd-33acc7224398 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:11.561135 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308ce6a9-d6ba-4a0a-92fd-33acc7224398 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:11.561413 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308ce6a9-d6ba-4a0a-92fd-33acc7224398 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:11.595010 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1336/2661] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:44:11 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:44:11.605947 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff4f172-7436-44b0-a987-17c62b0ac944 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:11.608209 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-2ff4f172-7436-44b0-a987-17c62b0ac944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:11.620072 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff4f172-7436-44b0-a987-17c62b0ac944 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:11.620234 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308ce6a9-d6ba-4a0a-92fd-33acc7224398 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:11.621910 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308ce6a9-d6ba-4a0a-92fd-33acc7224398 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:11.624799 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dd361b-3b24-4a87-b8f9-d3bfb0497588 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1be1e6173e374a60b9e1fbf51a1ab571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:11.625462 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dd361b-3b24-4a87-b8f9-d3bfb0497588 service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:11.625753 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dd361b-3b24-4a87-b8f9-d3bfb0497588 service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:11.626029 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dd361b-3b24-4a87-b8f9-d3bfb0497588 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:11.631399 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1326/2662] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:44:11 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 01:44:11.635106 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dd361b-3b24-4a87-b8f9-d3bfb0497588 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:11.636561 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dd361b-3b24-4a87-b8f9-d3bfb0497588 service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:11.646823 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1337/2663] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:44:11 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 01:44:12.257536 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308ce6a9-d6ba-4a0a-92fd-33acc7224398 service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:12.320628 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308ce6a9-d6ba-4a0a-92fd-33acc7224398 service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:12.325217 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32fc0bd-023e-4df1-b72a-ff2238a38a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:12.326454 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32fc0bd-023e-4df1-b72a-ff2238a38a41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:12.326874 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32fc0bd-023e-4df1-b72a-ff2238a38a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:12.327153 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32fc0bd-023e-4df1-b72a-ff2238a38a41 admin admin] PATH_INFO: `/v3/users/97776fedf80b4d07834007e5686978ce` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:12.328308 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32fc0bd-023e-4df1-b72a-ff2238a38a41 admin admin] RBAC: Authorizing `identity:delete_user(user_id=97776fedf80b4d07834007e5686978ce)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:12.330324 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32fc0bd-023e-4df1-b72a-ff2238a38a41 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:12.379664 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-f32fc0bd-023e-4df1-b72a-ff2238a38a41 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97776fedf80b4d07834007e5686978ce'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:12.380302 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-f32fc0bd-023e-4df1-b72a-ff2238a38a41 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97776fedf80b4d07834007e5686978ce'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:12.380949 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-f32fc0bd-023e-4df1-b72a-ff2238a38a41 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97776fedf80b4d07834007e5686978ce'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:12.394104 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:44:12.395315 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-f32fc0bd-023e-4df1-b72a-ff2238a38a41 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97776fedf80b4d07834007e5686978ce'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:12.404165 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-f32fc0bd-023e-4df1-b72a-ff2238a38a41 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97776fedf80b4d07834007e5686978ce'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:12.417498 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1327/2664] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Aug 6 01:44:12 2022] DELETE /identity/v3/users/97776fedf80b4d07834007e5686978ce => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:44:12.422048 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65b377b-889c-4cf9-82db-11c682aeed43 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:12.422544 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65b377b-889c-4cf9-82db-11c682aeed43 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:12.422656 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65b377b-889c-4cf9-82db-11c682aeed43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:12.441925 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-e65b377b-889c-4cf9-82db-11c682aeed43 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:44:12.443120 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-e65b377b-889c-4cf9-82db-11c682aeed43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:12.500138 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1338/2665] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:44:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:44:12.509739 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32fc0bd-023e-4df1-b72a-ff2238a38a41 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:12.511176 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-f32fc0bd-023e-4df1-b72a-ff2238a38a41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:12.569546 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32fc0bd-023e-4df1-b72a-ff2238a38a41 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:12.573162 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3727a523-4194-4aa8-bfb9-e6ce6dae8e39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:12.573778 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3727a523-4194-4aa8-bfb9-e6ce6dae8e39 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:12.574062 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3727a523-4194-4aa8-bfb9-e6ce6dae8e39 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:12.574341 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3727a523-4194-4aa8-bfb9-e6ce6dae8e39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:12.583738 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3727a523-4194-4aa8-bfb9-e6ce6dae8e39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:12.585181 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3727a523-4194-4aa8-bfb9-e6ce6dae8e39 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:12.594548 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1328/2666] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:44:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:44:12.868566 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65b377b-889c-4cf9-82db-11c682aeed43 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:12.870040 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-e65b377b-889c-4cf9-82db-11c682aeed43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:12.884237 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65b377b-889c-4cf9-82db-11c682aeed43 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:12.890625 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e681f6a4-3360-4dba-9204-c08033a5f975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:12.891444 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e681f6a4-3360-4dba-9204-c08033a5f975 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:12.891731 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e681f6a4-3360-4dba-9204-c08033a5f975 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:12.892129 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e681f6a4-3360-4dba-9204-c08033a5f975 admin admin] PATH_INFO: `/v3/projects/38dc5750595b4a43a66581fa56689bd3` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:12.893194 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e681f6a4-3360-4dba-9204-c08033a5f975 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38dc5750595b4a43a66581fa56689bd3)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:12.895150 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e681f6a4-3360-4dba-9204-c08033a5f975 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:12.943380 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-e681f6a4-3360-4dba-9204-c08033a5f975 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38dc5750595b4a43a66581fa56689bd3', 'request_id': 'req-e681f6a4-3360-4dba-9204-c08033a5f975'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:12.949438 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-e681f6a4-3360-4dba-9204-c08033a5f975 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38dc5750595b4a43a66581fa56689bd3', 'request_id': 'req-e681f6a4-3360-4dba-9204-c08033a5f975'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:12.955618 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-e681f6a4-3360-4dba-9204-c08033a5f975 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38dc5750595b4a43a66581fa56689bd3', 'request_id': 'req-e681f6a4-3360-4dba-9204-c08033a5f975'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:12.969689 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-e681f6a4-3360-4dba-9204-c08033a5f975 admin admin] The token cache is being invalidate because project 38dc5750595b4a43a66581fa56689bd3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:44:12.969689 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-e681f6a4-3360-4dba-9204-c08033a5f975 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:12.970309 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-e681f6a4-3360-4dba-9204-c08033a5f975 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:12.971820 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1339/2667] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Aug 6 01:44:12 2022] DELETE /identity/v3/projects/38dc5750595b4a43a66581fa56689bd3 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:44:12.975849 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3727a523-4194-4aa8-bfb9-e6ce6dae8e39 service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:12.985690 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3727a523-4194-4aa8-bfb9-e6ce6dae8e39 service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:13.046274 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d103f00-20c4-494d-a9a5-05d35f306e06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:13.046923 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d103f00-20c4-494d-a9a5-05d35f306e06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:13.047203 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d103f00-20c4-494d-a9a5-05d35f306e06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:13.047480 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d103f00-20c4-494d-a9a5-05d35f306e06 admin admin] PATH_INFO: `/v3/users/0118f08c0449477eaea77301906818f0` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:13.048648 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d103f00-20c4-494d-a9a5-05d35f306e06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0118f08c0449477eaea77301906818f0)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:13.050638 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d103f00-20c4-494d-a9a5-05d35f306e06 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:13.103545 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-4d103f00-20c4-494d-a9a5-05d35f306e06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0118f08c0449477eaea77301906818f0'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:13.104237 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-4d103f00-20c4-494d-a9a5-05d35f306e06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0118f08c0449477eaea77301906818f0'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:13.104854 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-4d103f00-20c4-494d-a9a5-05d35f306e06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0118f08c0449477eaea77301906818f0'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:13.120410 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-4d103f00-20c4-494d-a9a5-05d35f306e06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0118f08c0449477eaea77301906818f0'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:13.128056 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-4d103f00-20c4-494d-a9a5-05d35f306e06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0118f08c0449477eaea77301906818f0'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:13.142321 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1329/2668] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Aug 6 01:44:12 2022] DELETE /identity/v3/users/0118f08c0449477eaea77301906818f0 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:44:13.183696 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e681f6a4-3360-4dba-9204-c08033a5f975 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:13.186030 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-e681f6a4-3360-4dba-9204-c08033a5f975 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:13.360961 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:44:13.383003 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e681f6a4-3360-4dba-9204-c08033a5f975 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:13.436680 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba4cf24-e170-4ba6-a1f8-cdbfdecbcfff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1be1e6173e374a60b9e1fbf51a1ab571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:13.437482 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba4cf24-e170-4ba6-a1f8-cdbfdecbcfff service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:13.437766 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba4cf24-e170-4ba6-a1f8-cdbfdecbcfff service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:13.438072 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba4cf24-e170-4ba6-a1f8-cdbfdecbcfff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:13.440060 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-2ba4cf24-e170-4ba6-a1f8-cdbfdecbcfff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:13.453282 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d103f00-20c4-494d-a9a5-05d35f306e06 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:13.454828 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-4d103f00-20c4-494d-a9a5-05d35f306e06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:13.468468 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:44:13.504176 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba4cf24-e170-4ba6-a1f8-cdbfdecbcfff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:13.505649 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba4cf24-e170-4ba6-a1f8-cdbfdecbcfff service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:13.515481 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1340/2669] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:44:13 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 01:44:13.534435 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d103f00-20c4-494d-a9a5-05d35f306e06 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:13.539163 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601ec366-0fe3-44e0-be79-6d64cbfb1569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:13.539841 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601ec366-0fe3-44e0-be79-6d64cbfb1569 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:13.540208 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601ec366-0fe3-44e0-be79-6d64cbfb1569 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:13.540498 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601ec366-0fe3-44e0-be79-6d64cbfb1569 admin admin] PATH_INFO: `/v3/projects/244f8d9d29fd418abe5fea70dacca544` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:13.541704 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601ec366-0fe3-44e0-be79-6d64cbfb1569 admin admin] RBAC: Authorizing `identity:delete_project(project_id=244f8d9d29fd418abe5fea70dacca544)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:13.543619 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601ec366-0fe3-44e0-be79-6d64cbfb1569 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:13.595079 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-601ec366-0fe3-44e0-be79-6d64cbfb1569 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '244f8d9d29fd418abe5fea70dacca544', 'request_id': 'req-601ec366-0fe3-44e0-be79-6d64cbfb1569'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:13.601614 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-601ec366-0fe3-44e0-be79-6d64cbfb1569 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '244f8d9d29fd418abe5fea70dacca544', 'request_id': 'req-601ec366-0fe3-44e0-be79-6d64cbfb1569'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:13.608106 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-601ec366-0fe3-44e0-be79-6d64cbfb1569 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '244f8d9d29fd418abe5fea70dacca544', 'request_id': 'req-601ec366-0fe3-44e0-be79-6d64cbfb1569'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:13.621388 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-601ec366-0fe3-44e0-be79-6d64cbfb1569 admin admin] The token cache is being invalidate because project 244f8d9d29fd418abe5fea70dacca544 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:44:13.621919 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-601ec366-0fe3-44e0-be79-6d64cbfb1569 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:13.623296 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-601ec366-0fe3-44e0-be79-6d64cbfb1569 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:13.628462 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1330/2670] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Aug 6 01:44:13 2022] DELETE /identity/v3/projects/244f8d9d29fd418abe5fea70dacca544 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:44:13.630347 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce5a802-b924-45cb-9d3a-4c9141add6cd None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:13.630635 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce5a802-b924-45cb-9d3a-4c9141add6cd None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:13.630938 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce5a802-b924-45cb-9d3a-4c9141add6cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:13.652234 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-3ce5a802-b924-45cb-9d3a-4c9141add6cd None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:44:13.653509 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-3ce5a802-b924-45cb-9d3a-4c9141add6cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:13.715338 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1341/2671] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:44:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:44:13.724484 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b364fa-92e5-4636-9b61-06285324ec64 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:13.724880 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b364fa-92e5-4636-9b61-06285324ec64 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:13.725300 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b364fa-92e5-4636-9b61-06285324ec64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:13.746394 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-51b364fa-92e5-4636-9b61-06285324ec64 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:44:13.748273 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-51b364fa-92e5-4636-9b61-06285324ec64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:13.756228 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1331/2672] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:44:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:44:13.760533 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce5a802-b924-45cb-9d3a-4c9141add6cd None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:13.771292 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce5a802-b924-45cb-9d3a-4c9141add6cd None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:13.776323 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adac9bac-033a-4579-b972-83dc7abd329a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:13.776935 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adac9bac-033a-4579-b972-83dc7abd329a admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:13.777251 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adac9bac-033a-4579-b972-83dc7abd329a admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:13.777531 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adac9bac-033a-4579-b972-83dc7abd329a admin admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:13.778297 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adac9bac-033a-4579-b972-83dc7abd329a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:44:13.778720 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adac9bac-033a-4579-b972-83dc7abd329a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:13.780641 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adac9bac-033a-4579-b972-83dc7abd329a admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:13.793836 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1342/2673] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Aug 6 01:44:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:44:13.798825 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b364fa-92e5-4636-9b61-06285324ec64 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:13.807941 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b364fa-92e5-4636-9b61-06285324ec64 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:13.813280 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a43abc-932f-4bac-960e-8d0f95a53d0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:13.813885 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a43abc-932f-4bac-960e-8d0f95a53d0a admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:13.814164 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a43abc-932f-4bac-960e-8d0f95a53d0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:13.814436 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a43abc-932f-4bac-960e-8d0f95a53d0a admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:13.815089 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a43abc-932f-4bac-960e-8d0f95a53d0a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:13.816978 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a43abc-932f-4bac-960e-8d0f95a53d0a admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:13.844595 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1332/2674] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Aug 6 01:44:13 2022] POST /identity/v3/projects => generated 377 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:44:13.849035 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adac9bac-033a-4579-b972-83dc7abd329a admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:13.859874 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adac9bac-033a-4579-b972-83dc7abd329a admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:13.900819 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:44:13.922222 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0b1112-b455-4605-b149-d3e038dcc097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:13.922901 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0b1112-b455-4605-b149-d3e038dcc097 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:13.923216 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0b1112-b455-4605-b149-d3e038dcc097 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:13.923376 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0b1112-b455-4605-b149-d3e038dcc097 admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:13.924027 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0b1112-b455-4605-b149-d3e038dcc097 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:13.925969 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0b1112-b455-4605-b149-d3e038dcc097 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:13.946932 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1343/2675] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:44:13 2022] POST /identity/v3/users => generated 289 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:44:13.950912 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a43abc-932f-4bac-960e-8d0f95a53d0a admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:13.961362 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a43abc-932f-4bac-960e-8d0f95a53d0a admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:13.966811 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c32b65f-1949-410b-bc45-30c3cce6c0d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:13.967535 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c32b65f-1949-410b-bc45-30c3cce6c0d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:13.967822 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c32b65f-1949-410b-bc45-30c3cce6c0d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:13.968108 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c32b65f-1949-410b-bc45-30c3cce6c0d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:13.968810 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c32b65f-1949-410b-bc45-30c3cce6c0d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:13.970434 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c32b65f-1949-410b-bc45-30c3cce6c0d2 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:13.981169 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1333/2676] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:44:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:44:13.985985 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0b1112-b455-4605-b149-d3e038dcc097 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:13.996626 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0b1112-b455-4605-b149-d3e038dcc097 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:14.001675 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695bb97e-03f5-4b59-be04-38f11fa8ab3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:14.002298 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695bb97e-03f5-4b59-be04-38f11fa8ab3b admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:14.002611 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695bb97e-03f5-4b59-be04-38f11fa8ab3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:14.002890 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695bb97e-03f5-4b59-be04-38f11fa8ab3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:14.003587 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695bb97e-03f5-4b59-be04-38f11fa8ab3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:14.005223 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695bb97e-03f5-4b59-be04-38f11fa8ab3b admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:14.015454 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1344/2677] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:44:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:44:14.019756 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c32b65f-1949-410b-bc45-30c3cce6c0d2 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:14.030168 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c32b65f-1949-410b-bc45-30c3cce6c0d2 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:14.036041 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99972d45-5c06-4594-bcb9-cf5aaeb1b7cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:14.036761 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99972d45-5c06-4594-bcb9-cf5aaeb1b7cf admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:14.037074 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99972d45-5c06-4594-bcb9-cf5aaeb1b7cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:14.037485 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99972d45-5c06-4594-bcb9-cf5aaeb1b7cf admin admin] PATH_INFO: `/v3/projects/d45cecb47ddf422d8603fc4fd9993341/users/5832cc48bcaa4a3bab79ef52d24fce62/roles/61b3b93d37a047e4bb8369b46fe20db9` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:14.053197 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99972d45-5c06-4594-bcb9-cf5aaeb1b7cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=d45cecb47ddf422d8603fc4fd9993341, user_id=5832cc48bcaa4a3bab79ef52d24fce62, role_id=61b3b93d37a047e4bb8369b46fe20db9)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:14.055847 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99972d45-5c06-4594-bcb9-cf5aaeb1b7cf admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:14.072488 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1334/2678] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:44:14 2022] PUT /identity/v3/projects/d45cecb47ddf422d8603fc4fd9993341/users/5832cc48bcaa4a3bab79ef52d24fce62/roles/61b3b93d37a047e4bb8369b46fe20db9 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:44:14.081569 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695bb97e-03f5-4b59-be04-38f11fa8ab3b admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:14.081569 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-695bb97e-03f5-4b59-be04-38f11fa8ab3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:14.083546 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bcea14-f336-4cfa-bcc0-05399130be4a None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:14.083909 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bcea14-f336-4cfa-bcc0-05399130be4a None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:14.084263 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bcea14-f336-4cfa-bcc0-05399130be4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:14.112682 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-55bcea14-f336-4cfa-bcc0-05399130be4a None None] MFA Rules not processed for user `5832cc48bcaa4a3bab79ef52d24fce62`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:44:14.113933 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-55bcea14-f336-4cfa-bcc0-05399130be4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:14.130909 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:44:14.138433 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695bb97e-03f5-4b59-be04-38f11fa8ab3b admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:14.184139 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1335/2679] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:44:14 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:44:14.187537 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f665a5-38c0-4b4e-bb77-25a203370372 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1be1e6173e374a60b9e1fbf51a1ab571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:14.188154 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f665a5-38c0-4b4e-bb77-25a203370372 service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:14.188447 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f665a5-38c0-4b4e-bb77-25a203370372 service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:14.188714 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f665a5-38c0-4b4e-bb77-25a203370372 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:14.189399 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bcea14-f336-4cfa-bcc0-05399130be4a None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:14.190567 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-94f665a5-38c0-4b4e-bb77-25a203370372 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:14.252825 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bcea14-f336-4cfa-bcc0-05399130be4a None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:14.254855 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f665a5-38c0-4b4e-bb77-25a203370372 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:14.256439 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f665a5-38c0-4b4e-bb77-25a203370372 service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:14.257495 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8722784e-0bd6-4307-8163-5350f170a099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:14.258105 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8722784e-0bd6-4307-8163-5350f170a099 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:14.258383 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8722784e-0bd6-4307-8163-5350f170a099 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:14.258657 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8722784e-0bd6-4307-8163-5350f170a099 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:14.259312 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8722784e-0bd6-4307-8163-5350f170a099 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:14.261222 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8722784e-0bd6-4307-8163-5350f170a099 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:14.267112 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1345/2680] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:44:14 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:44:14.292296 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1336/2681] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Aug 6 01:44:14 2022] POST /identity/v3/projects => generated 377 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:44:14.293394 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f665a5-38c0-4b4e-bb77-25a203370372 service cinder] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:14.304092 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f665a5-38c0-4b4e-bb77-25a203370372 service cinder] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:14.327251 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:44:14.369099 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fd7426-9f03-49bd-b36d-98f4982b6945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:14.369724 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fd7426-9f03-49bd-b36d-98f4982b6945 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:14.370006 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fd7426-9f03-49bd-b36d-98f4982b6945 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:14.370278 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fd7426-9f03-49bd-b36d-98f4982b6945 admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:14.371213 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fd7426-9f03-49bd-b36d-98f4982b6945 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:14.373321 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fd7426-9f03-49bd-b36d-98f4982b6945 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:14.394459 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1346/2682] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:44:14 2022] POST /identity/v3/users => generated 289 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:44:14.398373 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8722784e-0bd6-4307-8163-5350f170a099 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:14.409500 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8722784e-0bd6-4307-8163-5350f170a099 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:14.414129 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ec6fd1-4933-469d-8efb-ff71b60589a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:14.414740 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ec6fd1-4933-469d-8efb-ff71b60589a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:14.415024 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ec6fd1-4933-469d-8efb-ff71b60589a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:14.415306 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ec6fd1-4933-469d-8efb-ff71b60589a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:14.415990 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ec6fd1-4933-469d-8efb-ff71b60589a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:14.417610 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ec6fd1-4933-469d-8efb-ff71b60589a7 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:14.427614 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1337/2683] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:44:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:44:14.431599 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fd7426-9f03-49bd-b36d-98f4982b6945 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:14.441862 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fd7426-9f03-49bd-b36d-98f4982b6945 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:14.446626 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a93038-e86a-4bb7-9a8e-7561e0da978f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:14.447244 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a93038-e86a-4bb7-9a8e-7561e0da978f admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:14.447505 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a93038-e86a-4bb7-9a8e-7561e0da978f admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:14.447775 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a93038-e86a-4bb7-9a8e-7561e0da978f admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:14.448464 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a93038-e86a-4bb7-9a8e-7561e0da978f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:14.450056 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a93038-e86a-4bb7-9a8e-7561e0da978f admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:14.460114 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1347/2684] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:44:14 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:44:14.464256 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ec6fd1-4933-469d-8efb-ff71b60589a7 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:14.474782 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ec6fd1-4933-469d-8efb-ff71b60589a7 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:14.479456 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c559dc64-158d-44f3-9bc0-c4a4d7050901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:14.480114 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c559dc64-158d-44f3-9bc0-c4a4d7050901 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:14.480404 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c559dc64-158d-44f3-9bc0-c4a4d7050901 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:14.480684 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c559dc64-158d-44f3-9bc0-c4a4d7050901 admin admin] PATH_INFO: `/v3/projects/d5fd314c9a2a49edbf8f3211f2359947/users/2d5cd4431e834c5c835cb07c48fea537/roles/61b3b93d37a047e4bb8369b46fe20db9` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:14.497020 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c559dc64-158d-44f3-9bc0-c4a4d7050901 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5fd314c9a2a49edbf8f3211f2359947, user_id=2d5cd4431e834c5c835cb07c48fea537, role_id=61b3b93d37a047e4bb8369b46fe20db9)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:14.499758 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c559dc64-158d-44f3-9bc0-c4a4d7050901 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:14.512769 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1338/2685] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:44:14 2022] PUT /identity/v3/projects/d5fd314c9a2a49edbf8f3211f2359947/users/2d5cd4431e834c5c835cb07c48fea537/roles/61b3b93d37a047e4bb8369b46fe20db9 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:44:14.527854 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b35f365-f80e-41bb-9451-62a42a7f45d0 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:14.528172 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b35f365-f80e-41bb-9451-62a42a7f45d0 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:14.528456 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b35f365-f80e-41bb-9451-62a42a7f45d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:14.557441 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-3b35f365-f80e-41bb-9451-62a42a7f45d0 None None] MFA Rules not processed for user `2d5cd4431e834c5c835cb07c48fea537`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:44:14.558592 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-3b35f365-f80e-41bb-9451-62a42a7f45d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:14.624702 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1348/2686] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:44:14 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:44:14.629416 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c559dc64-158d-44f3-9bc0-c4a4d7050901 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:14.690364 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c559dc64-158d-44f3-9bc0-c4a4d7050901 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:14.695071 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26526495-e1f6-4995-9919-bfea4329d995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:14.695704 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26526495-e1f6-4995-9919-bfea4329d995 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:14.695975 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26526495-e1f6-4995-9919-bfea4329d995 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:14.696255 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26526495-e1f6-4995-9919-bfea4329d995 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:14.696917 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26526495-e1f6-4995-9919-bfea4329d995 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:14.698843 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26526495-e1f6-4995-9919-bfea4329d995 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:14.713441 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:44:14.725676 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1339/2687] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Aug 6 01:44:14 2022] POST /identity/v3/projects => generated 375 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:44:14.729775 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b35f365-f80e-41bb-9451-62a42a7f45d0 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:14.740317 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b35f365-f80e-41bb-9451-62a42a7f45d0 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:14.795658 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c443d79-921c-4eeb-a1fd-0ff4f8e0205a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:14.796267 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c443d79-921c-4eeb-a1fd-0ff4f8e0205a admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:14.796547 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c443d79-921c-4eeb-a1fd-0ff4f8e0205a admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:14.796822 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c443d79-921c-4eeb-a1fd-0ff4f8e0205a admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:14.797519 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c443d79-921c-4eeb-a1fd-0ff4f8e0205a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:14.799426 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c443d79-921c-4eeb-a1fd-0ff4f8e0205a admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:14.820027 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1349/2688] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:44:14 2022] POST /identity/v3/users => generated 294 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:44:14.824231 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26526495-e1f6-4995-9919-bfea4329d995 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:14.835831 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26526495-e1f6-4995-9919-bfea4329d995 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:14.840630 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdb5c4e-ac1f-4fbc-8508-2498818730d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:14.841419 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdb5c4e-ac1f-4fbc-8508-2498818730d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:14.841701 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdb5c4e-ac1f-4fbc-8508-2498818730d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:14.841979 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdb5c4e-ac1f-4fbc-8508-2498818730d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:14.842723 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdb5c4e-ac1f-4fbc-8508-2498818730d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:14.844307 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdb5c4e-ac1f-4fbc-8508-2498818730d5 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:14.854950 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1340/2689] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:44:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 01:44:14.859397 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c443d79-921c-4eeb-a1fd-0ff4f8e0205a admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:14.869347 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c443d79-921c-4eeb-a1fd-0ff4f8e0205a admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:14.875034 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9e7b10-8b69-4ace-80d5-3376e3f7e59e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:14.875690 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9e7b10-8b69-4ace-80d5-3376e3f7e59e admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:14.875973 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9e7b10-8b69-4ace-80d5-3376e3f7e59e admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:14.876250 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9e7b10-8b69-4ace-80d5-3376e3f7e59e admin admin] PATH_INFO: `/v3/projects/b5ac10d028044e09a93b1703262eddce/users/006fb562a93b4e1c8bfa503632a3dd2f/roles/4559b9a0837249f895a13c70096f8d38` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:14.891112 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9e7b10-8b69-4ace-80d5-3376e3f7e59e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5ac10d028044e09a93b1703262eddce, user_id=006fb562a93b4e1c8bfa503632a3dd2f, role_id=4559b9a0837249f895a13c70096f8d38)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:14.893827 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9e7b10-8b69-4ace-80d5-3376e3f7e59e admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:14.901546 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:44:14.907718 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1350/2690] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:44:14 2022] PUT /identity/v3/projects/b5ac10d028044e09a93b1703262eddce/users/006fb562a93b4e1c8bfa503632a3dd2f/roles/4559b9a0837249f895a13c70096f8d38 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:44:14.911654 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdb5c4e-ac1f-4fbc-8508-2498818730d5 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:14.921563 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdb5c4e-ac1f-4fbc-8508-2498818730d5 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:14.978088 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b1e7d8-7023-454d-a9ce-32f6e5cfc554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:14.978718 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b1e7d8-7023-454d-a9ce-32f6e5cfc554 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:14.978984 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b1e7d8-7023-454d-a9ce-32f6e5cfc554 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:14.979260 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b1e7d8-7023-454d-a9ce-32f6e5cfc554 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:14.979932 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b1e7d8-7023-454d-a9ce-32f6e5cfc554 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:14.982166 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b1e7d8-7023-454d-a9ce-32f6e5cfc554 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:14.991896 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1341/2691] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:44:14 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:44:14.996314 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9e7b10-8b69-4ace-80d5-3376e3f7e59e admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:15.007023 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9e7b10-8b69-4ace-80d5-3376e3f7e59e admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:15.011551 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daae851-3545-4162-bb6f-91953a0ed00c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:15.012202 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daae851-3545-4162-bb6f-91953a0ed00c admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:15.012478 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daae851-3545-4162-bb6f-91953a0ed00c admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:15.012748 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daae851-3545-4162-bb6f-91953a0ed00c admin admin] PATH_INFO: `/v3/projects/b5ac10d028044e09a93b1703262eddce/users/006fb562a93b4e1c8bfa503632a3dd2f/roles/4559b9a0837249f895a13c70096f8d38` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:15.014701 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daae851-3545-4162-bb6f-91953a0ed00c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5ac10d028044e09a93b1703262eddce, user_id=006fb562a93b4e1c8bfa503632a3dd2f, role_id=4559b9a0837249f895a13c70096f8d38)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:15.017336 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daae851-3545-4162-bb6f-91953a0ed00c admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:15.027473 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1351/2692] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:44:14 2022] PUT /identity/v3/projects/b5ac10d028044e09a93b1703262eddce/users/006fb562a93b4e1c8bfa503632a3dd2f/roles/4559b9a0837249f895a13c70096f8d38 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:44:15.037485 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9824e477-6f4a-45c0-9a39-1ba6dcde4c31 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:15.037775 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9824e477-6f4a-45c0-9a39-1ba6dcde4c31 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:15.038051 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9824e477-6f4a-45c0-9a39-1ba6dcde4c31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:15.066899 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-9824e477-6f4a-45c0-9a39-1ba6dcde4c31 None None] MFA Rules not processed for user `006fb562a93b4e1c8bfa503632a3dd2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:44:15.068101 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-9824e477-6f4a-45c0-9a39-1ba6dcde4c31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:15.134034 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1342/2693] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:44:15 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:44:15.145567 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daae851-3545-4162-bb6f-91953a0ed00c admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:15.202343 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daae851-3545-4162-bb6f-91953a0ed00c admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:15.206104 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0828bc08-469f-4a8f-899d-e41eed356d83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1be1e6173e374a60b9e1fbf51a1ab571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:15.206720 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0828bc08-469f-4a8f-899d-e41eed356d83 service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:15.207004 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0828bc08-469f-4a8f-899d-e41eed356d83 service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:15.207301 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0828bc08-469f-4a8f-899d-e41eed356d83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:15.262658 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0828bc08-469f-4a8f-899d-e41eed356d83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:15.264224 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0828bc08-469f-4a8f-899d-e41eed356d83 service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:15.272927 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1352/2694] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:44:15 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:44:16.783188 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9824e477-6f4a-45c0-9a39-1ba6dcde4c31 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:16.786205 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-9824e477-6f4a-45c0-9a39-1ba6dcde4c31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:16.799551 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9824e477-6f4a-45c0-9a39-1ba6dcde4c31 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:16.803194 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74155672-cfc5-4546-9c52-b79d9186ed7f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1be1e6173e374a60b9e1fbf51a1ab571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:16.803807 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74155672-cfc5-4546-9c52-b79d9186ed7f service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:16.804093 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74155672-cfc5-4546-9c52-b79d9186ed7f service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:16.804378 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74155672-cfc5-4546-9c52-b79d9186ed7f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:16.863790 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74155672-cfc5-4546-9c52-b79d9186ed7f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:16.865263 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74155672-cfc5-4546-9c52-b79d9186ed7f service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:16.875220 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1343/2695] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:44:16 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:44:17.432375 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0828bc08-469f-4a8f-899d-e41eed356d83 service cinder] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:17.441111 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0828bc08-469f-4a8f-899d-e41eed356d83 service cinder] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:17.444657 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091f2386-346b-4bd4-b70d-0ea631a6651b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1be1e6173e374a60b9e1fbf51a1ab571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:17.445297 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091f2386-346b-4bd4-b70d-0ea631a6651b service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:17.445586 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091f2386-346b-4bd4-b70d-0ea631a6651b service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:17.445859 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091f2386-346b-4bd4-b70d-0ea631a6651b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:17.455629 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091f2386-346b-4bd4-b70d-0ea631a6651b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:17.457062 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091f2386-346b-4bd4-b70d-0ea631a6651b service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:17.467310 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1353/2696] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:44:17 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:44:18.155061 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74155672-cfc5-4546-9c52-b79d9186ed7f service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:18.156468 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-74155672-cfc5-4546-9c52-b79d9186ed7f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:18.161819 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:44:18.221940 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74155672-cfc5-4546-9c52-b79d9186ed7f service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:18.226464 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fa9d2f-b938-4811-9d1f-4e00bda63164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:18.227094 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fa9d2f-b938-4811-9d1f-4e00bda63164 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:18.227383 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fa9d2f-b938-4811-9d1f-4e00bda63164 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:18.227661 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fa9d2f-b938-4811-9d1f-4e00bda63164 admin admin] PATH_INFO: `/v3/users/90a4ea1f21e84e5a912055ad7cc80bdd` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:18.228829 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fa9d2f-b938-4811-9d1f-4e00bda63164 admin admin] RBAC: Authorizing `identity:delete_user(user_id=90a4ea1f21e84e5a912055ad7cc80bdd)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:18.230767 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fa9d2f-b938-4811-9d1f-4e00bda63164 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:18.276224 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-a7fa9d2f-b938-4811-9d1f-4e00bda63164 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90a4ea1f21e84e5a912055ad7cc80bdd'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:18.276869 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-a7fa9d2f-b938-4811-9d1f-4e00bda63164 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90a4ea1f21e84e5a912055ad7cc80bdd'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:18.277504 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-a7fa9d2f-b938-4811-9d1f-4e00bda63164 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90a4ea1f21e84e5a912055ad7cc80bdd'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:18.289195 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-a7fa9d2f-b938-4811-9d1f-4e00bda63164 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90a4ea1f21e84e5a912055ad7cc80bdd'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:18.296289 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-a7fa9d2f-b938-4811-9d1f-4e00bda63164 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90a4ea1f21e84e5a912055ad7cc80bdd'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:18.308273 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1344/2697] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Aug 6 01:44:18 2022] DELETE /identity/v3/users/90a4ea1f21e84e5a912055ad7cc80bdd => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:44:18.313415 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1a72f2-b7e5-4f2c-88e0-731d0eebff8d None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:18.313711 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1a72f2-b7e5-4f2c-88e0-731d0eebff8d None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:18.314081 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1a72f2-b7e5-4f2c-88e0-731d0eebff8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:18.332378 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-1c1a72f2-b7e5-4f2c-88e0-731d0eebff8d None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:44:18.333604 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-1c1a72f2-b7e5-4f2c-88e0-731d0eebff8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:18.392710 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1354/2698] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:44:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:44:18.402364 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fa9d2f-b938-4811-9d1f-4e00bda63164 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:18.404362 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-a7fa9d2f-b938-4811-9d1f-4e00bda63164 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:18.460309 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fa9d2f-b938-4811-9d1f-4e00bda63164 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:18.463956 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1913ccf-f2ed-4359-aae0-36f065ea73ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:18.464568 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1913ccf-f2ed-4359-aae0-36f065ea73ce service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:18.464853 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1913ccf-f2ed-4359-aae0-36f065ea73ce service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:18.465158 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1913ccf-f2ed-4359-aae0-36f065ea73ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:18.474651 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1913ccf-f2ed-4359-aae0-36f065ea73ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:18.476062 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1913ccf-f2ed-4359-aae0-36f065ea73ce service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:18.485934 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1345/2699] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:44:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:44:18.760778 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1a72f2-b7e5-4f2c-88e0-731d0eebff8d None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:18.762332 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-1c1a72f2-b7e5-4f2c-88e0-731d0eebff8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:18.784914 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1a72f2-b7e5-4f2c-88e0-731d0eebff8d None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:18.791459 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c152140e-2ff6-43d6-bd9b-6a912bc1f486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:18.797367 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c152140e-2ff6-43d6-bd9b-6a912bc1f486 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:18.797667 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c152140e-2ff6-43d6-bd9b-6a912bc1f486 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:18.797984 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c152140e-2ff6-43d6-bd9b-6a912bc1f486 admin admin] PATH_INFO: `/v3/projects/947dd3c9188d48d7a304d3150659f53a` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:18.799295 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c152140e-2ff6-43d6-bd9b-6a912bc1f486 admin admin] RBAC: Authorizing `identity:delete_project(project_id=947dd3c9188d48d7a304d3150659f53a)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:18.801323 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c152140e-2ff6-43d6-bd9b-6a912bc1f486 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:18.828600 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:44:18.856813 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-c152140e-2ff6-43d6-bd9b-6a912bc1f486 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '947dd3c9188d48d7a304d3150659f53a', 'request_id': 'req-c152140e-2ff6-43d6-bd9b-6a912bc1f486'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:18.863112 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-c152140e-2ff6-43d6-bd9b-6a912bc1f486 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '947dd3c9188d48d7a304d3150659f53a', 'request_id': 'req-c152140e-2ff6-43d6-bd9b-6a912bc1f486'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:18.881073 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-c152140e-2ff6-43d6-bd9b-6a912bc1f486 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '947dd3c9188d48d7a304d3150659f53a', 'request_id': 'req-c152140e-2ff6-43d6-bd9b-6a912bc1f486'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:18.884092 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-c152140e-2ff6-43d6-bd9b-6a912bc1f486 admin admin] The token cache is being invalidate because project 947dd3c9188d48d7a304d3150659f53a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:44:18.884455 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-c152140e-2ff6-43d6-bd9b-6a912bc1f486 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:18.885176 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-c152140e-2ff6-43d6-bd9b-6a912bc1f486 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:18.886763 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1355/2700] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Aug 6 01:44:18 2022] DELETE /identity/v3/projects/947dd3c9188d48d7a304d3150659f53a => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:44:18.891226 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1913ccf-f2ed-4359-aae0-36f065ea73ce service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:18.901123 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1913ccf-f2ed-4359-aae0-36f065ea73ce service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:18.921693 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:44:18.962598 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c740db-830f-4c33-91ab-d4a4c72c6e72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:18.963228 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c740db-830f-4c33-91ab-d4a4c72c6e72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:18.963510 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c740db-830f-4c33-91ab-d4a4c72c6e72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:18.963787 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c740db-830f-4c33-91ab-d4a4c72c6e72 admin admin] PATH_INFO: `/v3/users/8c14d03bf3164d71886a70c24772f503` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:18.964934 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c740db-830f-4c33-91ab-d4a4c72c6e72 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c14d03bf3164d71886a70c24772f503)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:18.967848 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c740db-830f-4c33-91ab-d4a4c72c6e72 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:19.012554 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-43c740db-830f-4c33-91ab-d4a4c72c6e72 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c14d03bf3164d71886a70c24772f503'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:19.013757 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-43c740db-830f-4c33-91ab-d4a4c72c6e72 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c14d03bf3164d71886a70c24772f503'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:19.014361 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-43c740db-830f-4c33-91ab-d4a4c72c6e72 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c14d03bf3164d71886a70c24772f503'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:19.026039 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-43c740db-830f-4c33-91ab-d4a4c72c6e72 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c14d03bf3164d71886a70c24772f503'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:19.033723 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-43c740db-830f-4c33-91ab-d4a4c72c6e72 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c14d03bf3164d71886a70c24772f503'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:19.046242 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1346/2701] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Aug 6 01:44:18 2022] DELETE /identity/v3/users/8c14d03bf3164d71886a70c24772f503 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:44:19.315460 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c152140e-2ff6-43d6-bd9b-6a912bc1f486 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:19.317058 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-c152140e-2ff6-43d6-bd9b-6a912bc1f486 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:19.330740 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c152140e-2ff6-43d6-bd9b-6a912bc1f486 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:19.388634 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d693087-4fe2-4818-b4e0-0ef481de88d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:19.389312 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d693087-4fe2-4818-b4e0-0ef481de88d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:19.389620 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d693087-4fe2-4818-b4e0-0ef481de88d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:19.389870 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d693087-4fe2-4818-b4e0-0ef481de88d2 admin admin] PATH_INFO: `/v3/projects/a5e8bc0340ca4134ac5f55e003a72545` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:19.391030 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d693087-4fe2-4818-b4e0-0ef481de88d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5e8bc0340ca4134ac5f55e003a72545)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:19.392928 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d693087-4fe2-4818-b4e0-0ef481de88d2 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:19.438803 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-1d693087-4fe2-4818-b4e0-0ef481de88d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5e8bc0340ca4134ac5f55e003a72545', 'request_id': 'req-1d693087-4fe2-4818-b4e0-0ef481de88d2'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:19.444917 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-1d693087-4fe2-4818-b4e0-0ef481de88d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5e8bc0340ca4134ac5f55e003a72545', 'request_id': 'req-1d693087-4fe2-4818-b4e0-0ef481de88d2'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:19.450639 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-1d693087-4fe2-4818-b4e0-0ef481de88d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5e8bc0340ca4134ac5f55e003a72545', 'request_id': 'req-1d693087-4fe2-4818-b4e0-0ef481de88d2'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:19.462729 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-1d693087-4fe2-4818-b4e0-0ef481de88d2 admin admin] The token cache is being invalidate because project a5e8bc0340ca4134ac5f55e003a72545 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:44:19.463019 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-1d693087-4fe2-4818-b4e0-0ef481de88d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:19.463641 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-1d693087-4fe2-4818-b4e0-0ef481de88d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:19.465145 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1356/2702] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Aug 6 01:44:19 2022] DELETE /identity/v3/projects/a5e8bc0340ca4134ac5f55e003a72545 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:44:19.469733 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c740db-830f-4c33-91ab-d4a4c72c6e72 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:19.471165 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-43c740db-830f-4c33-91ab-d4a4c72c6e72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:19.484341 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c740db-830f-4c33-91ab-d4a4c72c6e72 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:19.541592 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c8855e-7ecd-4c0e-bd14-915e3fed65f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:19.542231 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c8855e-7ecd-4c0e-bd14-915e3fed65f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:19.542675 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c8855e-7ecd-4c0e-bd14-915e3fed65f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:19.542951 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c8855e-7ecd-4c0e-bd14-915e3fed65f4 admin admin] PATH_INFO: `/v3/users/694c80a63b0c48d2956c8a379d24dd59` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:19.544099 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c8855e-7ecd-4c0e-bd14-915e3fed65f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=694c80a63b0c48d2956c8a379d24dd59)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:19.546042 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c8855e-7ecd-4c0e-bd14-915e3fed65f4 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:19.590533 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:44:19.592816 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-c6c8855e-7ecd-4c0e-bd14-915e3fed65f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '694c80a63b0c48d2956c8a379d24dd59'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:19.593657 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-c6c8855e-7ecd-4c0e-bd14-915e3fed65f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '694c80a63b0c48d2956c8a379d24dd59'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:19.594725 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-c6c8855e-7ecd-4c0e-bd14-915e3fed65f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '694c80a63b0c48d2956c8a379d24dd59'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:19.608718 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-c6c8855e-7ecd-4c0e-bd14-915e3fed65f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '694c80a63b0c48d2956c8a379d24dd59'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:19.615802 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-c6c8855e-7ecd-4c0e-bd14-915e3fed65f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '694c80a63b0c48d2956c8a379d24dd59'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:19.628652 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1347/2703] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Aug 6 01:44:19 2022] DELETE /identity/v3/users/694c80a63b0c48d2956c8a379d24dd59 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:44:19.902395 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d693087-4fe2-4818-b4e0-0ef481de88d2 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:19.904264 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-1d693087-4fe2-4818-b4e0-0ef481de88d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:19.922806 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d693087-4fe2-4818-b4e0-0ef481de88d2 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:19.966275 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:44:19.986991 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9db71f6-9de8-4700-bdf7-16b71eba45f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:19.987635 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9db71f6-9de8-4700-bdf7-16b71eba45f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:19.987936 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9db71f6-9de8-4700-bdf7-16b71eba45f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:19.988218 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9db71f6-9de8-4700-bdf7-16b71eba45f0 admin admin] PATH_INFO: `/v3/projects/6c53ce57303846acb7d840931f72b4f0` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:19.989408 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9db71f6-9de8-4700-bdf7-16b71eba45f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c53ce57303846acb7d840931f72b4f0)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:19.991341 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9db71f6-9de8-4700-bdf7-16b71eba45f0 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:20.044575 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a9db71f6-9de8-4700-bdf7-16b71eba45f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c53ce57303846acb7d840931f72b4f0', 'request_id': 'req-a9db71f6-9de8-4700-bdf7-16b71eba45f0'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:20.051001 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a9db71f6-9de8-4700-bdf7-16b71eba45f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c53ce57303846acb7d840931f72b4f0', 'request_id': 'req-a9db71f6-9de8-4700-bdf7-16b71eba45f0'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:20.057427 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a9db71f6-9de8-4700-bdf7-16b71eba45f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c53ce57303846acb7d840931f72b4f0', 'request_id': 'req-a9db71f6-9de8-4700-bdf7-16b71eba45f0'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:20.076148 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a9db71f6-9de8-4700-bdf7-16b71eba45f0 admin admin] The token cache is being invalidate because project 6c53ce57303846acb7d840931f72b4f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:44:20.076450 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a9db71f6-9de8-4700-bdf7-16b71eba45f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:20.077222 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a9db71f6-9de8-4700-bdf7-16b71eba45f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:20.078882 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1357/2704] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Aug 6 01:44:19 2022] DELETE /identity/v3/projects/6c53ce57303846acb7d840931f72b4f0 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:44:20.083680 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fc528c-9a09-4b9f-a9a1-c2b8867a9808 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:20.083974 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fc528c-9a09-4b9f-a9a1-c2b8867a9808 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:20.084252 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fc528c-9a09-4b9f-a9a1-c2b8867a9808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:20.103913 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-40fc528c-9a09-4b9f-a9a1-c2b8867a9808 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:44:20.105141 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-40fc528c-9a09-4b9f-a9a1-c2b8867a9808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:20.179688 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1348/2705] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:44:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:44:20.189104 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290a867f-ce67-4696-8b7d-020269df316a None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:20.189399 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290a867f-ce67-4696-8b7d-020269df316a None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:20.189821 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290a867f-ce67-4696-8b7d-020269df316a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:20.211616 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-290a867f-ce67-4696-8b7d-020269df316a None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:44:20.212815 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-290a867f-ce67-4696-8b7d-020269df316a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:20.222494 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1358/2706] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:44:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:44:20.227030 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fc528c-9a09-4b9f-a9a1-c2b8867a9808 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:20.241616 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fc528c-9a09-4b9f-a9a1-c2b8867a9808 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:20.246728 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180b9ccb-6de2-4f3e-854d-53a72acd4eae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:20.247993 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180b9ccb-6de2-4f3e-854d-53a72acd4eae admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:20.248304 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180b9ccb-6de2-4f3e-854d-53a72acd4eae admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:20.248601 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180b9ccb-6de2-4f3e-854d-53a72acd4eae admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:20.249480 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180b9ccb-6de2-4f3e-854d-53a72acd4eae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:44:20.249816 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180b9ccb-6de2-4f3e-854d-53a72acd4eae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:20.251414 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180b9ccb-6de2-4f3e-854d-53a72acd4eae admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:20.263451 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1349/2707] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Aug 6 01:44:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:44:20.268017 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290a867f-ce67-4696-8b7d-020269df316a None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:20.278069 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290a867f-ce67-4696-8b7d-020269df316a None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:20.283614 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276a4a36-c967-4add-bdac-142f10f7c235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:20.284239 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276a4a36-c967-4add-bdac-142f10f7c235 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:20.285219 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276a4a36-c967-4add-bdac-142f10f7c235 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:20.285504 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276a4a36-c967-4add-bdac-142f10f7c235 admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:20.286180 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276a4a36-c967-4add-bdac-142f10f7c235 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:20.288088 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276a4a36-c967-4add-bdac-142f10f7c235 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:20.315996 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1359/2708] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Aug 6 01:44:20 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:44:20.320283 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180b9ccb-6de2-4f3e-854d-53a72acd4eae admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:20.330310 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180b9ccb-6de2-4f3e-854d-53a72acd4eae admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:20.386786 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f1df1d-3232-4d34-8644-66a31c45a06b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:20.387401 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f1df1d-3232-4d34-8644-66a31c45a06b admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:20.387757 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f1df1d-3232-4d34-8644-66a31c45a06b admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:20.388083 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f1df1d-3232-4d34-8644-66a31c45a06b admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:20.388771 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f1df1d-3232-4d34-8644-66a31c45a06b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:20.390753 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f1df1d-3232-4d34-8644-66a31c45a06b admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:20.413213 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1350/2709] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:44:20 2022] POST /identity/v3/users => generated 286 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:44:20.417609 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276a4a36-c967-4add-bdac-142f10f7c235 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:20.428470 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276a4a36-c967-4add-bdac-142f10f7c235 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:20.433014 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c979170-7877-4f4a-8644-ee8502d75f54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:20.433642 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c979170-7877-4f4a-8644-ee8502d75f54 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:20.433919 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c979170-7877-4f4a-8644-ee8502d75f54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:20.434209 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c979170-7877-4f4a-8644-ee8502d75f54 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:20.435671 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c979170-7877-4f4a-8644-ee8502d75f54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:20.437294 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c979170-7877-4f4a-8644-ee8502d75f54 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:20.451397 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1360/2710] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:44:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:44:20.452572 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f1df1d-3232-4d34-8644-66a31c45a06b admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:20.462452 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f1df1d-3232-4d34-8644-66a31c45a06b admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:20.467116 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e143e4-516f-44aa-b236-9567783faab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:20.467722 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e143e4-516f-44aa-b236-9567783faab1 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:20.468004 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e143e4-516f-44aa-b236-9567783faab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:20.468275 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e143e4-516f-44aa-b236-9567783faab1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:20.468952 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e143e4-516f-44aa-b236-9567783faab1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:20.470666 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e143e4-516f-44aa-b236-9567783faab1 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:20.481168 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1351/2711] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:44:20 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:44:20.485858 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c979170-7877-4f4a-8644-ee8502d75f54 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:20.497972 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c979170-7877-4f4a-8644-ee8502d75f54 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:20.502578 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6293a177-967e-4d30-9907-ab84428430ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:20.503257 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6293a177-967e-4d30-9907-ab84428430ae admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:20.503536 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6293a177-967e-4d30-9907-ab84428430ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:20.503812 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6293a177-967e-4d30-9907-ab84428430ae admin admin] PATH_INFO: `/v3/projects/02aa058f34af417bb8ac0e6ce07ccf5b/users/c52a269e6385416481c12e92993ab8a1/roles/61b3b93d37a047e4bb8369b46fe20db9` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:20.519170 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6293a177-967e-4d30-9907-ab84428430ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=02aa058f34af417bb8ac0e6ce07ccf5b, user_id=c52a269e6385416481c12e92993ab8a1, role_id=61b3b93d37a047e4bb8369b46fe20db9)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:20.521942 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6293a177-967e-4d30-9907-ab84428430ae admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:20.534323 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1361/2712] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:44:20 2022] PUT /identity/v3/projects/02aa058f34af417bb8ac0e6ce07ccf5b/users/c52a269e6385416481c12e92993ab8a1/roles/61b3b93d37a047e4bb8369b46fe20db9 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:44:20.544922 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ab405a-b0b1-46ab-8744-b63b3ae527ce None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:20.545235 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ab405a-b0b1-46ab-8744-b63b3ae527ce None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:20.545502 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ab405a-b0b1-46ab-8744-b63b3ae527ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:20.574173 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-f2ab405a-b0b1-46ab-8744-b63b3ae527ce None None] MFA Rules not processed for user `c52a269e6385416481c12e92993ab8a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:44:20.575316 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-f2ab405a-b0b1-46ab-8744-b63b3ae527ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:20.591806 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:44:20.641306 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1352/2713] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:44:20 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:44:20.652708 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6293a177-967e-4d30-9907-ab84428430ae admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:20.655241 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-6293a177-967e-4d30-9907-ab84428430ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:20.710576 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6293a177-967e-4d30-9907-ab84428430ae admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:20.714210 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bacdc3-a280-41bb-a521-0b46c56047c0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abb76a5c8ef74bf2b18c56074daaad58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:20.714820 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bacdc3-a280-41bb-a521-0b46c56047c0 service glance] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:20.715117 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bacdc3-a280-41bb-a521-0b46c56047c0 service glance] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:20.715376 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bacdc3-a280-41bb-a521-0b46c56047c0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:20.724139 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bacdc3-a280-41bb-a521-0b46c56047c0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:20.725604 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bacdc3-a280-41bb-a521-0b46c56047c0 service glance] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:20.734804 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1362/2714] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:44:20 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:44:21.366287 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ab405a-b0b1-46ab-8744-b63b3ae527ce None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:21.367684 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-f2ab405a-b0b1-46ab-8744-b63b3ae527ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:21.435676 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ab405a-b0b1-46ab-8744-b63b3ae527ce None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:21.440182 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef817a9-c266-4bb6-9cfb-2bcdff1f1b4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:21.440820 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef817a9-c266-4bb6-9cfb-2bcdff1f1b4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:21.441137 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef817a9-c266-4bb6-9cfb-2bcdff1f1b4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:21.441418 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef817a9-c266-4bb6-9cfb-2bcdff1f1b4d admin admin] PATH_INFO: `/v3/users/5832cc48bcaa4a3bab79ef52d24fce62` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:21.442569 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef817a9-c266-4bb6-9cfb-2bcdff1f1b4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=5832cc48bcaa4a3bab79ef52d24fce62)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:21.444477 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef817a9-c266-4bb6-9cfb-2bcdff1f1b4d admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:21.490689 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-fef817a9-c266-4bb6-9cfb-2bcdff1f1b4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5832cc48bcaa4a3bab79ef52d24fce62'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:21.491647 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-fef817a9-c266-4bb6-9cfb-2bcdff1f1b4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5832cc48bcaa4a3bab79ef52d24fce62'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:21.492252 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-fef817a9-c266-4bb6-9cfb-2bcdff1f1b4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5832cc48bcaa4a3bab79ef52d24fce62'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:21.505526 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-fef817a9-c266-4bb6-9cfb-2bcdff1f1b4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5832cc48bcaa4a3bab79ef52d24fce62'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:21.514880 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-fef817a9-c266-4bb6-9cfb-2bcdff1f1b4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5832cc48bcaa4a3bab79ef52d24fce62'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:21.530854 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1353/2715] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Aug 6 01:44:21 2022] DELETE /identity/v3/users/5832cc48bcaa4a3bab79ef52d24fce62 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:44:21.536686 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae24107-3dd4-4956-85ac-9bd0200d314b None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:21.537258 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae24107-3dd4-4956-85ac-9bd0200d314b None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:21.537481 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae24107-3dd4-4956-85ac-9bd0200d314b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:21.565242 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-aae24107-3dd4-4956-85ac-9bd0200d314b None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:44:21.574231 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-aae24107-3dd4-4956-85ac-9bd0200d314b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:21.580288 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:44:21.642155 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1363/2716] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:44:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:44:21.651652 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef817a9-c266-4bb6-9cfb-2bcdff1f1b4d admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:21.653122 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-fef817a9-c266-4bb6-9cfb-2bcdff1f1b4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:21.698922 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:44:21.719293 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef817a9-c266-4bb6-9cfb-2bcdff1f1b4d admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:21.723191 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67eb4fad-3f2a-49b1-b86a-14dfdbec23c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:21.723821 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eb4fad-3f2a-49b1-b86a-14dfdbec23c0 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:21.724109 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eb4fad-3f2a-49b1-b86a-14dfdbec23c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:21.724390 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eb4fad-3f2a-49b1-b86a-14dfdbec23c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:21.734259 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67eb4fad-3f2a-49b1-b86a-14dfdbec23c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:21.735688 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67eb4fad-3f2a-49b1-b86a-14dfdbec23c0 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:21.746334 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1354/2717] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:44:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:44:22.038914 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae24107-3dd4-4956-85ac-9bd0200d314b None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:22.040336 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-aae24107-3dd4-4956-85ac-9bd0200d314b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:22.052396 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae24107-3dd4-4956-85ac-9bd0200d314b None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:22.057175 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c01677-331c-4fbd-b4a0-30d7f8d09a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:22.057854 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c01677-331c-4fbd-b4a0-30d7f8d09a50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:22.058141 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c01677-331c-4fbd-b4a0-30d7f8d09a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:22.058419 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c01677-331c-4fbd-b4a0-30d7f8d09a50 admin admin] PATH_INFO: `/v3/projects/d45cecb47ddf422d8603fc4fd9993341` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:22.059838 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c01677-331c-4fbd-b4a0-30d7f8d09a50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d45cecb47ddf422d8603fc4fd9993341)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:22.061870 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c01677-331c-4fbd-b4a0-30d7f8d09a50 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:22.116249 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-e2c01677-331c-4fbd-b4a0-30d7f8d09a50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd45cecb47ddf422d8603fc4fd9993341', 'request_id': 'req-e2c01677-331c-4fbd-b4a0-30d7f8d09a50'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:22.124117 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-e2c01677-331c-4fbd-b4a0-30d7f8d09a50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd45cecb47ddf422d8603fc4fd9993341', 'request_id': 'req-e2c01677-331c-4fbd-b4a0-30d7f8d09a50'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:22.130014 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-e2c01677-331c-4fbd-b4a0-30d7f8d09a50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd45cecb47ddf422d8603fc4fd9993341', 'request_id': 'req-e2c01677-331c-4fbd-b4a0-30d7f8d09a50'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:22.143473 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-e2c01677-331c-4fbd-b4a0-30d7f8d09a50 admin admin] The token cache is being invalidate because project d45cecb47ddf422d8603fc4fd9993341 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:44:22.143765 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-e2c01677-331c-4fbd-b4a0-30d7f8d09a50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:22.144469 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-e2c01677-331c-4fbd-b4a0-30d7f8d09a50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:22.146134 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1364/2718] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Aug 6 01:44:22 2022] DELETE /identity/v3/projects/d45cecb47ddf422d8603fc4fd9993341 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:44:22.150221 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67eb4fad-3f2a-49b1-b86a-14dfdbec23c0 service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:22.161560 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67eb4fad-3f2a-49b1-b86a-14dfdbec23c0 service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:22.233585 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580f007c-9181-41e6-8ff8-c4e963d508ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:22.234243 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580f007c-9181-41e6-8ff8-c4e963d508ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:22.234529 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580f007c-9181-41e6-8ff8-c4e963d508ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:22.234809 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580f007c-9181-41e6-8ff8-c4e963d508ca admin admin] PATH_INFO: `/v3/users/2d5cd4431e834c5c835cb07c48fea537` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:22.236013 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580f007c-9181-41e6-8ff8-c4e963d508ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d5cd4431e834c5c835cb07c48fea537)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:22.237984 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580f007c-9181-41e6-8ff8-c4e963d508ca admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:22.293173 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-580f007c-9181-41e6-8ff8-c4e963d508ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d5cd4431e834c5c835cb07c48fea537'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:22.293871 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-580f007c-9181-41e6-8ff8-c4e963d508ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d5cd4431e834c5c835cb07c48fea537'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:22.294498 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-580f007c-9181-41e6-8ff8-c4e963d508ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d5cd4431e834c5c835cb07c48fea537'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:22.306084 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-580f007c-9181-41e6-8ff8-c4e963d508ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d5cd4431e834c5c835cb07c48fea537'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:22.313543 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-580f007c-9181-41e6-8ff8-c4e963d508ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d5cd4431e834c5c835cb07c48fea537'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:22.326634 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1355/2719] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Aug 6 01:44:22 2022] DELETE /identity/v3/users/2d5cd4431e834c5c835cb07c48fea537 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:44:22.616752 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c01677-331c-4fbd-b4a0-30d7f8d09a50 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:22.618313 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-e2c01677-331c-4fbd-b4a0-30d7f8d09a50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:22.631821 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c01677-331c-4fbd-b4a0-30d7f8d09a50 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:22.693078 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bf8464-e0cd-4809-a3ab-359c95a08e6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:22.693719 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bf8464-e0cd-4809-a3ab-359c95a08e6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:22.694007 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bf8464-e0cd-4809-a3ab-359c95a08e6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:22.694268 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bf8464-e0cd-4809-a3ab-359c95a08e6f admin admin] PATH_INFO: `/v3/projects/d5fd314c9a2a49edbf8f3211f2359947` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:22.695406 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bf8464-e0cd-4809-a3ab-359c95a08e6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5fd314c9a2a49edbf8f3211f2359947)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:22.697355 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bf8464-e0cd-4809-a3ab-359c95a08e6f admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:22.735448 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:44:22.747950 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a2bf8464-e0cd-4809-a3ab-359c95a08e6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5fd314c9a2a49edbf8f3211f2359947', 'request_id': 'req-a2bf8464-e0cd-4809-a3ab-359c95a08e6f'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:22.754107 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a2bf8464-e0cd-4809-a3ab-359c95a08e6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5fd314c9a2a49edbf8f3211f2359947', 'request_id': 'req-a2bf8464-e0cd-4809-a3ab-359c95a08e6f'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:22.759536 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a2bf8464-e0cd-4809-a3ab-359c95a08e6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5fd314c9a2a49edbf8f3211f2359947', 'request_id': 'req-a2bf8464-e0cd-4809-a3ab-359c95a08e6f'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:22.771248 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a2bf8464-e0cd-4809-a3ab-359c95a08e6f admin admin] The token cache is being invalidate because project d5fd314c9a2a49edbf8f3211f2359947 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:44:22.771534 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a2bf8464-e0cd-4809-a3ab-359c95a08e6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:22.772141 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a2bf8464-e0cd-4809-a3ab-359c95a08e6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:22.773706 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1365/2720] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Aug 6 01:44:22 2022] DELETE /identity/v3/projects/d5fd314c9a2a49edbf8f3211f2359947 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:44:22.777780 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580f007c-9181-41e6-8ff8-c4e963d508ca admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:22.779277 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-580f007c-9181-41e6-8ff8-c4e963d508ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:22.792184 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580f007c-9181-41e6-8ff8-c4e963d508ca admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:22.848628 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96bf8a7-30b8-47ed-9a0e-f74d1b0ce6bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:22.849289 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96bf8a7-30b8-47ed-9a0e-f74d1b0ce6bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:22.849571 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96bf8a7-30b8-47ed-9a0e-f74d1b0ce6bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:22.849845 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96bf8a7-30b8-47ed-9a0e-f74d1b0ce6bb admin admin] PATH_INFO: `/v3/users/006fb562a93b4e1c8bfa503632a3dd2f` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:22.850990 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96bf8a7-30b8-47ed-9a0e-f74d1b0ce6bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=006fb562a93b4e1c8bfa503632a3dd2f)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:22.852918 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96bf8a7-30b8-47ed-9a0e-f74d1b0ce6bb admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:22.854413 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:44:22.911559 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-b96bf8a7-30b8-47ed-9a0e-f74d1b0ce6bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '006fb562a93b4e1c8bfa503632a3dd2f'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:22.912141 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-b96bf8a7-30b8-47ed-9a0e-f74d1b0ce6bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '006fb562a93b4e1c8bfa503632a3dd2f'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:22.912748 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-b96bf8a7-30b8-47ed-9a0e-f74d1b0ce6bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '006fb562a93b4e1c8bfa503632a3dd2f'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:22.925402 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-b96bf8a7-30b8-47ed-9a0e-f74d1b0ce6bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '006fb562a93b4e1c8bfa503632a3dd2f'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:22.932144 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-b96bf8a7-30b8-47ed-9a0e-f74d1b0ce6bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '006fb562a93b4e1c8bfa503632a3dd2f'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:22.945392 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1356/2721] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Aug 6 01:44:22 2022] DELETE /identity/v3/users/006fb562a93b4e1c8bfa503632a3dd2f => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:44:23.227154 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bf8464-e0cd-4809-a3ab-359c95a08e6f admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:23.228994 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-a2bf8464-e0cd-4809-a3ab-359c95a08e6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:23.243644 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bf8464-e0cd-4809-a3ab-359c95a08e6f admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:23.302820 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7b7e5f-059d-4ccb-a919-f936c4ad2ebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:23.303464 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7b7e5f-059d-4ccb-a919-f936c4ad2ebd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:23.303745 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7b7e5f-059d-4ccb-a919-f936c4ad2ebd admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:23.304021 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7b7e5f-059d-4ccb-a919-f936c4ad2ebd admin admin] PATH_INFO: `/v3/projects/b5ac10d028044e09a93b1703262eddce` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:23.305230 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7b7e5f-059d-4ccb-a919-f936c4ad2ebd admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5ac10d028044e09a93b1703262eddce)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:23.307127 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7b7e5f-059d-4ccb-a919-f936c4ad2ebd admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:23.359029 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-bb7b7e5f-059d-4ccb-a919-f936c4ad2ebd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5ac10d028044e09a93b1703262eddce', 'request_id': 'req-bb7b7e5f-059d-4ccb-a919-f936c4ad2ebd'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:23.365251 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-bb7b7e5f-059d-4ccb-a919-f936c4ad2ebd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5ac10d028044e09a93b1703262eddce', 'request_id': 'req-bb7b7e5f-059d-4ccb-a919-f936c4ad2ebd'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:23.372553 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-bb7b7e5f-059d-4ccb-a919-f936c4ad2ebd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5ac10d028044e09a93b1703262eddce', 'request_id': 'req-bb7b7e5f-059d-4ccb-a919-f936c4ad2ebd'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:23.386375 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-bb7b7e5f-059d-4ccb-a919-f936c4ad2ebd admin admin] The token cache is being invalidate because project b5ac10d028044e09a93b1703262eddce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:44:23.386667 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-bb7b7e5f-059d-4ccb-a919-f936c4ad2ebd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:23.387287 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-bb7b7e5f-059d-4ccb-a919-f936c4ad2ebd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:23.389437 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1366/2722] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Aug 6 01:44:23 2022] DELETE /identity/v3/projects/b5ac10d028044e09a93b1703262eddce => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:44:23.394304 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b16398-f765-4f18-bb0d-bc153a6bce15 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:23.394596 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b16398-f765-4f18-bb0d-bc153a6bce15 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:23.394875 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b16398-f765-4f18-bb0d-bc153a6bce15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:23.415367 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-84b16398-f765-4f18-bb0d-bc153a6bce15 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:44:23.416580 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-84b16398-f765-4f18-bb0d-bc153a6bce15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:23.474975 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1357/2723] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:44:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:44:23.482931 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a04a5dd-1ce9-493d-bf50-00dcd6f34846 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:23.483221 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a04a5dd-1ce9-493d-bf50-00dcd6f34846 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:23.483529 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a04a5dd-1ce9-493d-bf50-00dcd6f34846 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:23.503215 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-5a04a5dd-1ce9-493d-bf50-00dcd6f34846 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:44:23.504416 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-5a04a5dd-1ce9-493d-bf50-00dcd6f34846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:23.512428 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1367/2724] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:44:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:44:23.517201 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b16398-f765-4f18-bb0d-bc153a6bce15 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:23.526909 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b16398-f765-4f18-bb0d-bc153a6bce15 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:23.532157 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e4f6d6-1778-46a5-8f92-af595662165a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:23.532760 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e4f6d6-1778-46a5-8f92-af595662165a admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:23.533035 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e4f6d6-1778-46a5-8f92-af595662165a admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:23.533339 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e4f6d6-1778-46a5-8f92-af595662165a admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:23.534026 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e4f6d6-1778-46a5-8f92-af595662165a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:44:23.534326 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e4f6d6-1778-46a5-8f92-af595662165a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:23.535900 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e4f6d6-1778-46a5-8f92-af595662165a admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:23.548546 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1358/2725] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Aug 6 01:44:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:44:23.552962 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a04a5dd-1ce9-493d-bf50-00dcd6f34846 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:23.562681 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a04a5dd-1ce9-493d-bf50-00dcd6f34846 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:23.567315 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15688c7-8e18-4a27-8443-e3650a16ea4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:23.567921 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15688c7-8e18-4a27-8443-e3650a16ea4e admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:23.568212 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15688c7-8e18-4a27-8443-e3650a16ea4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:23.568476 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15688c7-8e18-4a27-8443-e3650a16ea4e admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:23.569194 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15688c7-8e18-4a27-8443-e3650a16ea4e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:23.571086 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15688c7-8e18-4a27-8443-e3650a16ea4e admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:23.605022 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1368/2726] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Aug 6 01:44:23 2022] POST /identity/v3/projects => generated 373 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:44:23.609914 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e4f6d6-1778-46a5-8f92-af595662165a admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:23.619924 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e4f6d6-1778-46a5-8f92-af595662165a admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:23.677876 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a58ad53-db4e-49f4-bdd3-4a3e6c67e4cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:23.678498 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a58ad53-db4e-49f4-bdd3-4a3e6c67e4cb admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:23.678786 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a58ad53-db4e-49f4-bdd3-4a3e6c67e4cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:23.679067 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a58ad53-db4e-49f4-bdd3-4a3e6c67e4cb admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:23.679739 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a58ad53-db4e-49f4-bdd3-4a3e6c67e4cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:23.681717 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a58ad53-db4e-49f4-bdd3-4a3e6c67e4cb admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:23.692348 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:44:23.704292 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1359/2727] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:44:23 2022] POST /identity/v3/users => generated 287 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:44:23.708522 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15688c7-8e18-4a27-8443-e3650a16ea4e admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:23.719286 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15688c7-8e18-4a27-8443-e3650a16ea4e admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:23.723854 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6713615a-c9be-4c0d-9175-e74a7397f594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:23.724472 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6713615a-c9be-4c0d-9175-e74a7397f594 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:23.724732 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6713615a-c9be-4c0d-9175-e74a7397f594 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:23.725067 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6713615a-c9be-4c0d-9175-e74a7397f594 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:23.725747 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6713615a-c9be-4c0d-9175-e74a7397f594 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:23.727318 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6713615a-c9be-4c0d-9175-e74a7397f594 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:23.737610 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1369/2728] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:44:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:44:23.741645 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a58ad53-db4e-49f4-bdd3-4a3e6c67e4cb admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:23.751752 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a58ad53-db4e-49f4-bdd3-4a3e6c67e4cb admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:23.756464 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11b5411-7788-4102-b8ec-c0126572f411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:23.757092 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11b5411-7788-4102-b8ec-c0126572f411 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:23.757378 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11b5411-7788-4102-b8ec-c0126572f411 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:23.757649 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11b5411-7788-4102-b8ec-c0126572f411 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:23.759372 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11b5411-7788-4102-b8ec-c0126572f411 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:23.760978 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11b5411-7788-4102-b8ec-c0126572f411 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:23.771745 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1360/2729] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:44:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:44:23.775966 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6713615a-c9be-4c0d-9175-e74a7397f594 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:23.786335 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6713615a-c9be-4c0d-9175-e74a7397f594 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:23.790984 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51a492d-a327-46fa-88f8-4545f7c16251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:23.791637 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51a492d-a327-46fa-88f8-4545f7c16251 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:23.791922 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51a492d-a327-46fa-88f8-4545f7c16251 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:23.792199 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51a492d-a327-46fa-88f8-4545f7c16251 admin admin] PATH_INFO: `/v3/projects/6ca94b8e3b5646bdb21a9a60bee4cc8d/users/6ad659c1e25d4c5eb52456936acf88ab/roles/61b3b93d37a047e4bb8369b46fe20db9` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:23.803778 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:44:23.808472 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51a492d-a327-46fa-88f8-4545f7c16251 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ca94b8e3b5646bdb21a9a60bee4cc8d, user_id=6ad659c1e25d4c5eb52456936acf88ab, role_id=61b3b93d37a047e4bb8369b46fe20db9)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:23.812281 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51a492d-a327-46fa-88f8-4545f7c16251 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:23.833876 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1370/2730] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:44:23 2022] PUT /identity/v3/projects/6ca94b8e3b5646bdb21a9a60bee4cc8d/users/6ad659c1e25d4c5eb52456936acf88ab/roles/61b3b93d37a047e4bb8369b46fe20db9 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:44:23.901525 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e224a4d5-84e2-4f6e-b9fe-827d0edae89f None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:23.902071 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e224a4d5-84e2-4f6e-b9fe-827d0edae89f None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:23.902071 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e224a4d5-84e2-4f6e-b9fe-827d0edae89f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:23.930202 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-e224a4d5-84e2-4f6e-b9fe-827d0edae89f None None] MFA Rules not processed for user `6ad659c1e25d4c5eb52456936acf88ab`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:44:23.931421 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-e224a4d5-84e2-4f6e-b9fe-827d0edae89f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:23.995366 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1361/2731] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:44:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:44:24.007308 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51a492d-a327-46fa-88f8-4545f7c16251 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:24.009126 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-b51a492d-a327-46fa-88f8-4545f7c16251 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:24.069791 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51a492d-a327-46fa-88f8-4545f7c16251 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:24.074136 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c908a393-4d50-4db8-83a6-1f82fc41c26e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1be1e6173e374a60b9e1fbf51a1ab571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:24.074930 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c908a393-4d50-4db8-83a6-1f82fc41c26e service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:24.075794 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c908a393-4d50-4db8-83a6-1f82fc41c26e service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:24.076071 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c908a393-4d50-4db8-83a6-1f82fc41c26e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:24.085926 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c908a393-4d50-4db8-83a6-1f82fc41c26e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:24.087362 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c908a393-4d50-4db8-83a6-1f82fc41c26e service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:24.097431 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1371/2732] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:44:24 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:44:33.992374 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=39 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:44:34.093256 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:44:35.033395 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e224a4d5-84e2-4f6e-b9fe-827d0edae89f None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:35.095528 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e224a4d5-84e2-4f6e-b9fe-827d0edae89f None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:35.100048 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1846531-5500-4d22-9510-aa55ad3e4106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:35.100684 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1846531-5500-4d22-9510-aa55ad3e4106 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:35.101000 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1846531-5500-4d22-9510-aa55ad3e4106 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:35.101315 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1846531-5500-4d22-9510-aa55ad3e4106 admin admin] PATH_INFO: `/v3/users/6ad659c1e25d4c5eb52456936acf88ab` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:35.102479 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1846531-5500-4d22-9510-aa55ad3e4106 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ad659c1e25d4c5eb52456936acf88ab)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:35.104398 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1846531-5500-4d22-9510-aa55ad3e4106 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:35.150042 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-c1846531-5500-4d22-9510-aa55ad3e4106 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ad659c1e25d4c5eb52456936acf88ab'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:35.150712 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-c1846531-5500-4d22-9510-aa55ad3e4106 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ad659c1e25d4c5eb52456936acf88ab'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:35.151313 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-c1846531-5500-4d22-9510-aa55ad3e4106 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ad659c1e25d4c5eb52456936acf88ab'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:35.163909 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-c1846531-5500-4d22-9510-aa55ad3e4106 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ad659c1e25d4c5eb52456936acf88ab'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:35.171335 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-c1846531-5500-4d22-9510-aa55ad3e4106 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ad659c1e25d4c5eb52456936acf88ab'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:35.183709 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1362/2733] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Aug 6 01:44:35 2022] DELETE /identity/v3/users/6ad659c1e25d4c5eb52456936acf88ab => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:44:35.188434 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8197200d-0d49-491b-ac80-5228c0d2da07 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:35.188895 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8197200d-0d49-491b-ac80-5228c0d2da07 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:35.188993 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8197200d-0d49-491b-ac80-5228c0d2da07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:35.207702 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-8197200d-0d49-491b-ac80-5228c0d2da07 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:44:35.208902 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-8197200d-0d49-491b-ac80-5228c0d2da07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:35.267049 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1372/2734] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:44:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:44:35.276843 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1846531-5500-4d22-9510-aa55ad3e4106 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:35.278285 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-c1846531-5500-4d22-9510-aa55ad3e4106 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:35.337628 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1846531-5500-4d22-9510-aa55ad3e4106 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:35.341850 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e25b31-8ad1-43ba-ad03-562e31220517 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:35.342466 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e25b31-8ad1-43ba-ad03-562e31220517 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:35.342746 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e25b31-8ad1-43ba-ad03-562e31220517 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:35.343022 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e25b31-8ad1-43ba-ad03-562e31220517 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:35.351954 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e25b31-8ad1-43ba-ad03-562e31220517 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:35.353351 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e25b31-8ad1-43ba-ad03-562e31220517 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:35.362667 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1363/2735] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:44:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:44:35.631427 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8197200d-0d49-491b-ac80-5228c0d2da07 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:35.632885 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-8197200d-0d49-491b-ac80-5228c0d2da07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:35.645434 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8197200d-0d49-491b-ac80-5228c0d2da07 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:35.650055 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2d5a25-1595-4049-8de3-7a4cfc980225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:35.650690 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2d5a25-1595-4049-8de3-7a4cfc980225 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:35.650975 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2d5a25-1595-4049-8de3-7a4cfc980225 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:35.651250 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2d5a25-1595-4049-8de3-7a4cfc980225 admin admin] PATH_INFO: `/v3/projects/6ca94b8e3b5646bdb21a9a60bee4cc8d` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:35.652409 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2d5a25-1595-4049-8de3-7a4cfc980225 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ca94b8e3b5646bdb21a9a60bee4cc8d)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:35.654355 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2d5a25-1595-4049-8de3-7a4cfc980225 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:35.702501 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-0e2d5a25-1595-4049-8de3-7a4cfc980225 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ca94b8e3b5646bdb21a9a60bee4cc8d', 'request_id': 'req-0e2d5a25-1595-4049-8de3-7a4cfc980225'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:35.708434 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-0e2d5a25-1595-4049-8de3-7a4cfc980225 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ca94b8e3b5646bdb21a9a60bee4cc8d', 'request_id': 'req-0e2d5a25-1595-4049-8de3-7a4cfc980225'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:35.713862 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-0e2d5a25-1595-4049-8de3-7a4cfc980225 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ca94b8e3b5646bdb21a9a60bee4cc8d', 'request_id': 'req-0e2d5a25-1595-4049-8de3-7a4cfc980225'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:35.726477 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-0e2d5a25-1595-4049-8de3-7a4cfc980225 admin admin] The token cache is being invalidate because project 6ca94b8e3b5646bdb21a9a60bee4cc8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:44:35.726770 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-0e2d5a25-1595-4049-8de3-7a4cfc980225 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:35.727374 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-0e2d5a25-1595-4049-8de3-7a4cfc980225 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:35.728858 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1373/2736] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Aug 6 01:44:35 2022] DELETE /identity/v3/projects/6ca94b8e3b5646bdb21a9a60bee4cc8d => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:44:35.734871 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd9bd59-f2fc-454e-84d5-018b2689a70f None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:35.735160 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd9bd59-f2fc-454e-84d5-018b2689a70f None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:35.735437 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd9bd59-f2fc-454e-84d5-018b2689a70f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:35.755872 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-cbd9bd59-f2fc-454e-84d5-018b2689a70f None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:44:35.757099 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-cbd9bd59-f2fc-454e-84d5-018b2689a70f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:35.774534 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:44:35.818947 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1364/2737] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:44:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:44:35.826553 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d736da2-f9b8-44db-9ec4-47318ddc784e None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:35.826839 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d736da2-f9b8-44db-9ec4-47318ddc784e None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:35.827121 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d736da2-f9b8-44db-9ec4-47318ddc784e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:35.846990 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-0d736da2-f9b8-44db-9ec4-47318ddc784e None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:44:35.848185 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-0d736da2-f9b8-44db-9ec4-47318ddc784e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:35.856426 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1374/2738] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:44:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:44:35.861182 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd9bd59-f2fc-454e-84d5-018b2689a70f None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:35.871235 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd9bd59-f2fc-454e-84d5-018b2689a70f None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:35.876162 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232ef094-2853-43a1-986e-fa7dde5c45b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:35.876713 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232ef094-2853-43a1-986e-fa7dde5c45b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:35.876993 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232ef094-2853-43a1-986e-fa7dde5c45b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:35.877388 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232ef094-2853-43a1-986e-fa7dde5c45b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:35.878081 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232ef094-2853-43a1-986e-fa7dde5c45b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:44:35.878388 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232ef094-2853-43a1-986e-fa7dde5c45b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:35.880079 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232ef094-2853-43a1-986e-fa7dde5c45b6 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:35.895944 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1365/2739] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Aug 6 01:44:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:44:35.900201 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d736da2-f9b8-44db-9ec4-47318ddc784e None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:35.909938 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d736da2-f9b8-44db-9ec4-47318ddc784e None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:35.914936 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106cd009-5949-48b6-bfa2-79a52957871d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:35.915545 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106cd009-5949-48b6-bfa2-79a52957871d admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:35.915825 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106cd009-5949-48b6-bfa2-79a52957871d admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:35.916087 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106cd009-5949-48b6-bfa2-79a52957871d admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:35.916739 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106cd009-5949-48b6-bfa2-79a52957871d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:35.918731 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106cd009-5949-48b6-bfa2-79a52957871d admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:35.947136 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1375/2740] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Aug 6 01:44:35 2022] POST /identity/v3/projects => generated 377 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:44:35.951413 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232ef094-2853-43a1-986e-fa7dde5c45b6 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:35.961239 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232ef094-2853-43a1-986e-fa7dde5c45b6 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:36.017730 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aea3a6-b17f-45eb-a213-6ecdbe7864a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:36.018344 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aea3a6-b17f-45eb-a213-6ecdbe7864a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:36.018632 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aea3a6-b17f-45eb-a213-6ecdbe7864a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:36.018912 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aea3a6-b17f-45eb-a213-6ecdbe7864a4 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:36.019584 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79aea3a6-b17f-45eb-a213-6ecdbe7864a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:36.021544 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79aea3a6-b17f-45eb-a213-6ecdbe7864a4 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:36.042388 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1366/2741] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:44:35 2022] POST /identity/v3/users => generated 289 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:44:36.046729 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106cd009-5949-48b6-bfa2-79a52957871d admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:36.056965 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106cd009-5949-48b6-bfa2-79a52957871d admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:36.062049 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1c4ad8-64a9-4d6f-ba46-9d9edf0d5719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:36.062683 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1c4ad8-64a9-4d6f-ba46-9d9edf0d5719 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:36.062968 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1c4ad8-64a9-4d6f-ba46-9d9edf0d5719 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:36.063242 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1c4ad8-64a9-4d6f-ba46-9d9edf0d5719 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:36.063939 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1c4ad8-64a9-4d6f-ba46-9d9edf0d5719 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:36.065557 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1c4ad8-64a9-4d6f-ba46-9d9edf0d5719 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:36.075434 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1376/2742] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:44:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:44:36.079815 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aea3a6-b17f-45eb-a213-6ecdbe7864a4 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:36.089995 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aea3a6-b17f-45eb-a213-6ecdbe7864a4 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:36.095377 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d799962-6025-4d40-81fe-3f4434417eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:36.095990 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d799962-6025-4d40-81fe-3f4434417eba admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:36.096276 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d799962-6025-4d40-81fe-3f4434417eba admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:36.096559 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d799962-6025-4d40-81fe-3f4434417eba admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:36.097286 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d799962-6025-4d40-81fe-3f4434417eba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:36.098888 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d799962-6025-4d40-81fe-3f4434417eba admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:36.109686 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1367/2743] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:44:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:44:36.113518 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1c4ad8-64a9-4d6f-ba46-9d9edf0d5719 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:36.123789 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1c4ad8-64a9-4d6f-ba46-9d9edf0d5719 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:36.128440 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f2196d-59c7-4e0f-bab4-9cd61bffe44a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:36.129031 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f2196d-59c7-4e0f-bab4-9cd61bffe44a admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:36.129344 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f2196d-59c7-4e0f-bab4-9cd61bffe44a admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:36.129622 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f2196d-59c7-4e0f-bab4-9cd61bffe44a admin admin] PATH_INFO: `/v3/projects/3bea2c3f927a4bccb59065a8380b98c9/users/6ac05cb43bb146cbb0a548ff3148a77a/roles/61b3b93d37a047e4bb8369b46fe20db9` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:36.145418 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f2196d-59c7-4e0f-bab4-9cd61bffe44a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bea2c3f927a4bccb59065a8380b98c9, user_id=6ac05cb43bb146cbb0a548ff3148a77a, role_id=61b3b93d37a047e4bb8369b46fe20db9)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:36.148062 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f2196d-59c7-4e0f-bab4-9cd61bffe44a admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:36.161341 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1377/2744] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:44:36 2022] PUT /identity/v3/projects/3bea2c3f927a4bccb59065a8380b98c9/users/6ac05cb43bb146cbb0a548ff3148a77a/roles/61b3b93d37a047e4bb8369b46fe20db9 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:44:36.166275 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef25987d-7b39-4a5d-b01a-5bb08912f93e None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:36.166569 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef25987d-7b39-4a5d-b01a-5bb08912f93e None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:36.166860 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef25987d-7b39-4a5d-b01a-5bb08912f93e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:36.185425 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-ef25987d-7b39-4a5d-b01a-5bb08912f93e None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:44:36.186643 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-ef25987d-7b39-4a5d-b01a-5bb08912f93e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:36.246595 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1368/2745] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:44:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:44:36.256233 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f2196d-59c7-4e0f-bab4-9cd61bffe44a admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:36.257953 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-55f2196d-59c7-4e0f-bab4-9cd61bffe44a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:36.280650 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:44:36.316504 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f2196d-59c7-4e0f-bab4-9cd61bffe44a admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:36.320100 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc61c9a-3793-4c78-b8a7-0c52ff10ee0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:36.320705 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc61c9a-3793-4c78-b8a7-0c52ff10ee0b service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:36.320986 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc61c9a-3793-4c78-b8a7-0c52ff10ee0b service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:36.321286 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc61c9a-3793-4c78-b8a7-0c52ff10ee0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:36.330538 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc61c9a-3793-4c78-b8a7-0c52ff10ee0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:36.331957 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc61c9a-3793-4c78-b8a7-0c52ff10ee0b service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:36.341207 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1378/2746] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:44:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:44:40.313807 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2a5171-4aad-44e1-a812-5f7fa95f0eba None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:40.314307 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2a5171-4aad-44e1-a812-5f7fa95f0eba None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:40.314419 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2a5171-4aad-44e1-a812-5f7fa95f0eba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:40.342256 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-5f2a5171-4aad-44e1-a812-5f7fa95f0eba None None] MFA Rules not processed for user `6ac05cb43bb146cbb0a548ff3148a77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:44:40.343521 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-5f2a5171-4aad-44e1-a812-5f7fa95f0eba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:40.359994 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:44:40.418967 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1369/2747] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:44:40 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:44:40.424311 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc61c9a-3793-4c78-b8a7-0c52ff10ee0b service neutron] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:40.434568 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc61c9a-3793-4c78-b8a7-0c52ff10ee0b service neutron] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:40.439543 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b280b9d0-bb8c-44f5-9461-0478faa38523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:40.440241 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b280b9d0-bb8c-44f5-9461-0478faa38523 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:40.440593 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b280b9d0-bb8c-44f5-9461-0478faa38523 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:40.440940 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b280b9d0-bb8c-44f5-9461-0478faa38523 admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:40.441713 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b280b9d0-bb8c-44f5-9461-0478faa38523 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:40.443622 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b280b9d0-bb8c-44f5-9461-0478faa38523 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:40.471486 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1379/2748] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Aug 6 01:44:40 2022] POST /identity/v3/projects => generated 379 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:44:40.475791 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2a5171-4aad-44e1-a812-5f7fa95f0eba None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:40.487774 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2a5171-4aad-44e1-a812-5f7fa95f0eba None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:40.546799 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79739ba9-ebca-459d-9a32-ab9f6d1af61b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:40.547477 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79739ba9-ebca-459d-9a32-ab9f6d1af61b admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:40.547700 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79739ba9-ebca-459d-9a32-ab9f6d1af61b admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:40.547977 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79739ba9-ebca-459d-9a32-ab9f6d1af61b admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:40.548648 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79739ba9-ebca-459d-9a32-ab9f6d1af61b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:40.550644 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79739ba9-ebca-459d-9a32-ab9f6d1af61b admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:40.586558 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1370/2749] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:44:40 2022] POST /identity/v3/users => generated 296 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:44:40.586777 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b280b9d0-bb8c-44f5-9461-0478faa38523 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:40.598532 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b280b9d0-bb8c-44f5-9461-0478faa38523 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:40.603317 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e70765-eafd-4aa3-82d6-bc09f62dff1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:40.603927 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e70765-eafd-4aa3-82d6-bc09f62dff1f admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:40.604209 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e70765-eafd-4aa3-82d6-bc09f62dff1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:40.604487 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e70765-eafd-4aa3-82d6-bc09f62dff1f admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:40.605249 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e70765-eafd-4aa3-82d6-bc09f62dff1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:40.607088 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e70765-eafd-4aa3-82d6-bc09f62dff1f admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:40.620384 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1380/2750] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:44:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:44:40.624867 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79739ba9-ebca-459d-9a32-ab9f6d1af61b admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:40.635182 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79739ba9-ebca-459d-9a32-ab9f6d1af61b admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:40.640787 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14331dc7-27c0-4d02-8c50-5938365fe5a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:40.641511 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14331dc7-27c0-4d02-8c50-5938365fe5a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:40.641794 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14331dc7-27c0-4d02-8c50-5938365fe5a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:40.642073 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14331dc7-27c0-4d02-8c50-5938365fe5a5 admin admin] PATH_INFO: `/v3/projects/c9a8221dc2ea41ea8bcb8a829cfc454f/users/5083dc0c8e644ad48342790c8c099c7a/roles/4559b9a0837249f895a13c70096f8d38` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:40.658841 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14331dc7-27c0-4d02-8c50-5938365fe5a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9a8221dc2ea41ea8bcb8a829cfc454f, user_id=5083dc0c8e644ad48342790c8c099c7a, role_id=4559b9a0837249f895a13c70096f8d38)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:40.661609 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14331dc7-27c0-4d02-8c50-5938365fe5a5 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:40.676019 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1371/2751] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:44:40 2022] PUT /identity/v3/projects/c9a8221dc2ea41ea8bcb8a829cfc454f/users/5083dc0c8e644ad48342790c8c099c7a/roles/4559b9a0837249f895a13c70096f8d38 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:44:40.680128 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e70765-eafd-4aa3-82d6-bc09f62dff1f admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:40.692701 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e70765-eafd-4aa3-82d6-bc09f62dff1f admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:40.752353 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b99a908-be95-4742-b813-f6fa2b8b9f66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:40.752977 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b99a908-be95-4742-b813-f6fa2b8b9f66 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:40.753294 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b99a908-be95-4742-b813-f6fa2b8b9f66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:40.753562 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b99a908-be95-4742-b813-f6fa2b8b9f66 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:40.754256 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b99a908-be95-4742-b813-f6fa2b8b9f66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:40.755884 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b99a908-be95-4742-b813-f6fa2b8b9f66 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:40.769973 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1381/2752] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:44:40 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:44:40.771695 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14331dc7-27c0-4d02-8c50-5938365fe5a5 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:40.783703 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14331dc7-27c0-4d02-8c50-5938365fe5a5 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:40.788525 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f76d48a-02e9-4a1a-a7a3-87e3a841a0f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:40.789252 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f76d48a-02e9-4a1a-a7a3-87e3a841a0f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:40.789531 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f76d48a-02e9-4a1a-a7a3-87e3a841a0f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:40.789804 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f76d48a-02e9-4a1a-a7a3-87e3a841a0f0 admin admin] PATH_INFO: `/v3/projects/c9a8221dc2ea41ea8bcb8a829cfc454f/users/5083dc0c8e644ad48342790c8c099c7a/roles/4559b9a0837249f895a13c70096f8d38` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:40.791896 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f76d48a-02e9-4a1a-a7a3-87e3a841a0f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9a8221dc2ea41ea8bcb8a829cfc454f, user_id=5083dc0c8e644ad48342790c8c099c7a, role_id=4559b9a0837249f895a13c70096f8d38)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:40.794726 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f76d48a-02e9-4a1a-a7a3-87e3a841a0f0 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:40.806242 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1372/2753] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:44:40 2022] PUT /identity/v3/projects/c9a8221dc2ea41ea8bcb8a829cfc454f/users/5083dc0c8e644ad48342790c8c099c7a/roles/4559b9a0837249f895a13c70096f8d38 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:44:45.377677 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c13f320-1851-41c0-85e8-6674688440b5 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:45.378238 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c13f320-1851-41c0-85e8-6674688440b5 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:45.378238 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c13f320-1851-41c0-85e8-6674688440b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:45.409595 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-1c13f320-1851-41c0-85e8-6674688440b5 None None] MFA Rules not processed for user `5083dc0c8e644ad48342790c8c099c7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:44:45.411035 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-1c13f320-1851-41c0-85e8-6674688440b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:45.497085 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1382/2754] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:44:45 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:44:45.512945 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f76d48a-02e9-4a1a-a7a3-87e3a841a0f0 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:45.515494 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-0f76d48a-02e9-4a1a-a7a3-87e3a841a0f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:45.619870 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f76d48a-02e9-4a1a-a7a3-87e3a841a0f0 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:45.629676 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3384f40d-1572-4550-a769-4bbc893ac30f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87adc1d5bc240e99f95f6595b350be4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:45.630603 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3384f40d-1572-4550-a769-4bbc893ac30f service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:45.630989 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3384f40d-1572-4550-a769-4bbc893ac30f service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:45.631381 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3384f40d-1572-4550-a769-4bbc893ac30f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:45.637901 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:44:45.726683 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3384f40d-1572-4550-a769-4bbc893ac30f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:45.733780 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3384f40d-1572-4550-a769-4bbc893ac30f service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:45.745643 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1373/2755] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:44:45 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:44:55.493963 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=94,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:44:55.740488 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=24 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:44:58.313436 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c13f320-1851-41c0-85e8-6674688440b5 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:58.314849 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-1c13f320-1851-41c0-85e8-6674688440b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:58.378761 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c13f320-1851-41c0-85e8-6674688440b5 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:58.383481 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed2c269-e88a-4aa6-a158-d7887751822d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:58.384137 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed2c269-e88a-4aa6-a158-d7887751822d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:58.384423 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed2c269-e88a-4aa6-a158-d7887751822d admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:58.384702 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed2c269-e88a-4aa6-a158-d7887751822d admin admin] PATH_INFO: `/v3/users/c52a269e6385416481c12e92993ab8a1` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:58.385910 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed2c269-e88a-4aa6-a158-d7887751822d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c52a269e6385416481c12e92993ab8a1)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:58.388465 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed2c269-e88a-4aa6-a158-d7887751822d admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:58.436377 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-eed2c269-e88a-4aa6-a158-d7887751822d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c52a269e6385416481c12e92993ab8a1'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:58.443517 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-eed2c269-e88a-4aa6-a158-d7887751822d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c52a269e6385416481c12e92993ab8a1'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:58.444186 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-eed2c269-e88a-4aa6-a158-d7887751822d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c52a269e6385416481c12e92993ab8a1'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:58.444799 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-eed2c269-e88a-4aa6-a158-d7887751822d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c52a269e6385416481c12e92993ab8a1'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:58.456968 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-eed2c269-e88a-4aa6-a158-d7887751822d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c52a269e6385416481c12e92993ab8a1'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:58.469941 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1383/2756] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Aug 6 01:44:58 2022] DELETE /identity/v3/users/c52a269e6385416481c12e92993ab8a1 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:44:58.475384 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f310032-160d-4971-8100-b36c744304b0 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:58.475984 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f310032-160d-4971-8100-b36c744304b0 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:58.476171 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f310032-160d-4971-8100-b36c744304b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:58.497790 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-6f310032-160d-4971-8100-b36c744304b0 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:44:58.499338 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-6f310032-160d-4971-8100-b36c744304b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:58.560569 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1374/2757] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:44:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:44:58.570140 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed2c269-e88a-4aa6-a158-d7887751822d admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:58.571513 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-eed2c269-e88a-4aa6-a158-d7887751822d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:58.632136 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed2c269-e88a-4aa6-a158-d7887751822d admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:58.635788 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b28adb-c038-4575-b39b-a024fdcc33b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:58.636402 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b28adb-c038-4575-b39b-a024fdcc33b8 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:58.636679 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b28adb-c038-4575-b39b-a024fdcc33b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:58.636947 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b28adb-c038-4575-b39b-a024fdcc33b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:58.647984 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b28adb-c038-4575-b39b-a024fdcc33b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:58.649869 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b28adb-c038-4575-b39b-a024fdcc33b8 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:58.661364 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1384/2758] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:44:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 01:44:58.933084 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f310032-160d-4971-8100-b36c744304b0 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:44:58.934845 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-6f310032-160d-4971-8100-b36c744304b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:44:58.947327 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f310032-160d-4971-8100-b36c744304b0 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:44:58.952086 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987864a9-f9bb-462a-a943-df502fa55d1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:44:58.952732 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987864a9-f9bb-462a-a943-df502fa55d1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:44:58.953011 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987864a9-f9bb-462a-a943-df502fa55d1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:44:58.953329 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987864a9-f9bb-462a-a943-df502fa55d1c admin admin] PATH_INFO: `/v3/projects/02aa058f34af417bb8ac0e6ce07ccf5b` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:44:58.954496 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987864a9-f9bb-462a-a943-df502fa55d1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=02aa058f34af417bb8ac0e6ce07ccf5b)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:44:58.956405 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987864a9-f9bb-462a-a943-df502fa55d1c admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:44:59.005804 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-987864a9-f9bb-462a-a943-df502fa55d1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02aa058f34af417bb8ac0e6ce07ccf5b', 'request_id': 'req-987864a9-f9bb-462a-a943-df502fa55d1c'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:59.011847 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-987864a9-f9bb-462a-a943-df502fa55d1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02aa058f34af417bb8ac0e6ce07ccf5b', 'request_id': 'req-987864a9-f9bb-462a-a943-df502fa55d1c'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:59.017364 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-987864a9-f9bb-462a-a943-df502fa55d1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02aa058f34af417bb8ac0e6ce07ccf5b', 'request_id': 'req-987864a9-f9bb-462a-a943-df502fa55d1c'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:59.029828 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-987864a9-f9bb-462a-a943-df502fa55d1c admin admin] The token cache is being invalidate because project 02aa058f34af417bb8ac0e6ce07ccf5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:44:59.030121 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-987864a9-f9bb-462a-a943-df502fa55d1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:59.030751 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-987864a9-f9bb-462a-a943-df502fa55d1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:44:59.032260 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1375/2759] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Aug 6 01:44:58 2022] DELETE /identity/v3/projects/02aa058f34af417bb8ac0e6ce07ccf5b => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:45:08.656382 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:45:08.944537 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b28adb-c038-4575-b39b-a024fdcc33b8 service neutron] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:45:08.946211 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-15b28adb-c038-4575-b39b-a024fdcc33b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:45:09.026124 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:45:09.039417 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b28adb-c038-4575-b39b-a024fdcc33b8 service neutron] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:45:09.110364 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f118f2-47c4-45fa-80f7-77dee45c7da8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87adc1d5bc240e99f95f6595b350be4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:45:09.111770 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f118f2-47c4-45fa-80f7-77dee45c7da8 service nova] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:45:09.111770 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f118f2-47c4-45fa-80f7-77dee45c7da8 service nova] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:45:09.112022 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f118f2-47c4-45fa-80f7-77dee45c7da8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:45:09.114304 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-17f118f2-47c4-45fa-80f7-77dee45c7da8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:45:09.131167 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f118f2-47c4-45fa-80f7-77dee45c7da8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:45:09.132121 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f118f2-47c4-45fa-80f7-77dee45c7da8 service nova] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:45:09.146593 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1385/2760] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:45:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:45:10.598188 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f128d86-d9d6-45d5-8e31-569e2a91841b None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:45:10.598713 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f128d86-d9d6-45d5-8e31-569e2a91841b None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:45:10.598878 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f128d86-d9d6-45d5-8e31-569e2a91841b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:45:10.618614 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-1f128d86-d9d6-45d5-8e31-569e2a91841b None None] MFA Rules not processed for user `6ac05cb43bb146cbb0a548ff3148a77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:45:10.619842 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-1f128d86-d9d6-45d5-8e31-569e2a91841b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:45:10.695905 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1376/2761] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:45:10 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:45:19.141127 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=62 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:45:20.686072 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=25 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:45:25.207702 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851cd41e-00c0-40c1-8723-63ae80c1ee67 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:45:25.208239 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851cd41e-00c0-40c1-8723-63ae80c1ee67 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:45:25.208348 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851cd41e-00c0-40c1-8723-63ae80c1ee67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:45:25.226989 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-851cd41e-00c0-40c1-8723-63ae80c1ee67 None None] MFA Rules not processed for user `d2235317fe8d4d7ebae7bb5b80dfd7e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:45:25.228204 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-851cd41e-00c0-40c1-8723-63ae80c1ee67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:45:25.284339 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1386/2762] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:45:25 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:45:25.833314 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f128d86-d9d6-45d5-8e31-569e2a91841b None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:45:25.834716 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-1f128d86-d9d6-45d5-8e31-569e2a91841b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:45:25.891276 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f128d86-d9d6-45d5-8e31-569e2a91841b None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:45:25.894858 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff48813-d265-4fd8-a5f1-eb96e7d58c5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:45:25.895467 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff48813-d265-4fd8-a5f1-eb96e7d58c5e service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:45:25.895745 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff48813-d265-4fd8-a5f1-eb96e7d58c5e service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:45:25.896017 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff48813-d265-4fd8-a5f1-eb96e7d58c5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:45:25.897791 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-dff48813-d265-4fd8-a5f1-eb96e7d58c5e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:45:25.907671 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff48813-d265-4fd8-a5f1-eb96e7d58c5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:45:25.909146 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff48813-d265-4fd8-a5f1-eb96e7d58c5e service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:45:25.916975 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1377/2763] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:45:25 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:45:33.020186 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851cd41e-00c0-40c1-8723-63ae80c1ee67 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:45:33.021796 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-851cd41e-00c0-40c1-8723-63ae80c1ee67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:45:33.040160 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851cd41e-00c0-40c1-8723-63ae80c1ee67 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:45:33.046637 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b8e3bf-612e-461c-9209-589e560e14f3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87adc1d5bc240e99f95f6595b350be4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:45:33.047500 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b8e3bf-612e-461c-9209-589e560e14f3 service nova] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:45:33.047781 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b8e3bf-612e-461c-9209-589e560e14f3 service nova] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:45:33.048193 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b8e3bf-612e-461c-9209-589e560e14f3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:45:33.050156 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-69b8e3bf-612e-461c-9209-589e560e14f3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:45:33.064337 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b8e3bf-612e-461c-9209-589e560e14f3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:45:33.065970 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b8e3bf-612e-461c-9209-589e560e14f3 service nova] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:45:33.077561 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1387/2764] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:45:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:45:35.193154 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff48813-d265-4fd8-a5f1-eb96e7d58c5e service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:45:35.202603 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff48813-d265-4fd8-a5f1-eb96e7d58c5e service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:45:35.206256 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00d6418-1397-483d-b80b-65102493aa30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:45:35.206911 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00d6418-1397-483d-b80b-65102493aa30 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:45:35.207194 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00d6418-1397-483d-b80b-65102493aa30 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:45:35.207461 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00d6418-1397-483d-b80b-65102493aa30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:45:35.209281 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-e00d6418-1397-483d-b80b-65102493aa30 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:45:35.271076 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00d6418-1397-483d-b80b-65102493aa30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:45:35.272494 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00d6418-1397-483d-b80b-65102493aa30 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:45:35.282487 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1378/2765] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:45:35 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:45:43.072367 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=31 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:45:45.277613 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=56 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:46:56.248113 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b8e3bf-612e-461c-9209-589e560e14f3 service nova] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:46:56.259309 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b8e3bf-612e-461c-9209-589e560e14f3 service nova] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:46:56.264826 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761fb549-ecf7-4cdc-8e4d-908e768f3007 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87adc1d5bc240e99f95f6595b350be4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:46:56.265464 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761fb549-ecf7-4cdc-8e4d-908e768f3007 service nova] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:46:56.265749 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761fb549-ecf7-4cdc-8e4d-908e768f3007 service nova] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:46:56.266025 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761fb549-ecf7-4cdc-8e4d-908e768f3007 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:46:56.267756 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-761fb549-ecf7-4cdc-8e4d-908e768f3007 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:46:56.277969 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761fb549-ecf7-4cdc-8e4d-908e768f3007 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:46:56.279538 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761fb549-ecf7-4cdc-8e4d-908e768f3007 service nova] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:46:56.288916 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1388/2766] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:46:56 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:47:04.849938 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00d6418-1397-483d-b80b-65102493aa30 service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:47:04.851420 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-e00d6418-1397-483d-b80b-65102493aa30 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:47:04.924138 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00d6418-1397-483d-b80b-65102493aa30 service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:47:04.929269 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca57123-762b-4ff7-ac54-b9c0854ce555 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa3b93ddd9464a2cba724d67b56b477a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:47:04.929887 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca57123-762b-4ff7-ac54-b9c0854ce555 service placement] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:47:04.930223 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca57123-762b-4ff7-ac54-b9c0854ce555 service placement] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:47:04.930507 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca57123-762b-4ff7-ac54-b9c0854ce555 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:47:04.932363 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-9ca57123-762b-4ff7-ac54-b9c0854ce555 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:47:04.944165 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca57123-762b-4ff7-ac54-b9c0854ce555 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:47:04.945798 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca57123-762b-4ff7-ac54-b9c0854ce555 service placement] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:47:04.955793 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1379/2767] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:47:04 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:47:06.284283 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=6 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:47:13.742053 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1efec56-a04d-4602-9590-6cce73299430 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:47:13.742684 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1efec56-a04d-4602-9590-6cce73299430 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:47:13.742684 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1efec56-a04d-4602-9590-6cce73299430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:47:13.761487 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-b1efec56-a04d-4602-9590-6cce73299430 None None] MFA Rules not processed for user `d2235317fe8d4d7ebae7bb5b80dfd7e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:47:13.762688 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-b1efec56-a04d-4602-9590-6cce73299430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:47:13.770632 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1389/2768] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:47:13 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:47:14.951380 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=32 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:47:23.756896 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:47:52.555356 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca57123-762b-4ff7-ac54-b9c0854ce555 service placement] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:47:52.565181 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca57123-762b-4ff7-ac54-b9c0854ce555 service placement] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:47:52.568908 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254b8313-2539-4aa4-9034-9f1a589e290b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:47:52.569540 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254b8313-2539-4aa4-9034-9f1a589e290b service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:47:52.569818 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254b8313-2539-4aa4-9034-9f1a589e290b service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:47:52.570162 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254b8313-2539-4aa4-9034-9f1a589e290b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:47:52.572019 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-254b8313-2539-4aa4-9034-9f1a589e290b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:47:52.585725 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254b8313-2539-4aa4-9034-9f1a589e290b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:47:52.587127 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254b8313-2539-4aa4-9034-9f1a589e290b service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:47:52.598073 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1380/2769] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:47:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:47:53.779345 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1efec56-a04d-4602-9590-6cce73299430 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:47:53.790006 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1efec56-a04d-4602-9590-6cce73299430 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:47:53.793537 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3271b5-786b-43a9-8b19-c652e569a96e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:47:53.794141 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3271b5-786b-43a9-8b19-c652e569a96e service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:47:53.794416 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3271b5-786b-43a9-8b19-c652e569a96e service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:47:53.794683 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3271b5-786b-43a9-8b19-c652e569a96e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:47:53.796383 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-3c3271b5-786b-43a9-8b19-c652e569a96e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:47:53.810533 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3271b5-786b-43a9-8b19-c652e569a96e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:47:53.811937 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3271b5-786b-43a9-8b19-c652e569a96e service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:47:53.825327 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1390/2770] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:47:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:47:56.120352 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b88005-b3cf-46d4-b38c-064d7a38d449 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:47:56.120901 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b88005-b3cf-46d4-b38c-064d7a38d449 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:47:56.121006 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b88005-b3cf-46d4-b38c-064d7a38d449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:47:56.141706 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-b2b88005-b3cf-46d4-b38c-064d7a38d449 None None] MFA Rules not processed for user `6ac05cb43bb146cbb0a548ff3148a77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:47:56.142906 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-b2b88005-b3cf-46d4-b38c-064d7a38d449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:47:56.151498 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1381/2771] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:47:56 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:48:03.819147 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=6 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:48:06.136292 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=9 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:49:01.521863 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3271b5-786b-43a9-8b19-c652e569a96e service neutron] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:49:01.531531 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3271b5-786b-43a9-8b19-c652e569a96e service neutron] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:49:01.535307 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1327844b-24bc-43a8-81e1-0dcc5cdba675 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:49:01.535921 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1327844b-24bc-43a8-81e1-0dcc5cdba675 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:49:01.536204 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1327844b-24bc-43a8-81e1-0dcc5cdba675 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:49:01.536479 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1327844b-24bc-43a8-81e1-0dcc5cdba675 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:49:01.538270 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-1327844b-24bc-43a8-81e1-0dcc5cdba675 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:49:01.600433 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1327844b-24bc-43a8-81e1-0dcc5cdba675 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:49:01.601904 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1327844b-24bc-43a8-81e1-0dcc5cdba675 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:49:01.611775 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1391/2772] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:49:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:49:04.431809 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b88005-b3cf-46d4-b38c-064d7a38d449 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:49:04.445795 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b88005-b3cf-46d4-b38c-064d7a38d449 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:49:04.451061 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ccc537-8fa2-40f1-af54-f72d8d5e7e3c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa3b93ddd9464a2cba724d67b56b477a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:49:04.451994 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ccc537-8fa2-40f1-af54-f72d8d5e7e3c service placement] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:49:04.452273 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ccc537-8fa2-40f1-af54-f72d8d5e7e3c service placement] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:49:04.452544 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ccc537-8fa2-40f1-af54-f72d8d5e7e3c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:49:04.454426 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-b2ccc537-8fa2-40f1-af54-f72d8d5e7e3c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:49:04.468082 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ccc537-8fa2-40f1-af54-f72d8d5e7e3c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:49:04.470046 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ccc537-8fa2-40f1-af54-f72d8d5e7e3c service placement] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:49:04.483363 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1382/2773] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:49:04 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:49:04.513416 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1327844b-24bc-43a8-81e1-0dcc5cdba675 service neutron] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:49:04.514955 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-1327844b-24bc-43a8-81e1-0dcc5cdba675 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:49:04.530248 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1327844b-24bc-43a8-81e1-0dcc5cdba675 service neutron] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:49:04.538931 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d933793d-2220-4a77-b510-7ebf213919f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:49:04.540162 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d933793d-2220-4a77-b510-7ebf213919f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:49:04.540484 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d933793d-2220-4a77-b510-7ebf213919f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:49:04.540785 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d933793d-2220-4a77-b510-7ebf213919f6 admin admin] PATH_INFO: `/v3/users/d2235317fe8d4d7ebae7bb5b80dfd7e6` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:49:04.542027 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d933793d-2220-4a77-b510-7ebf213919f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2235317fe8d4d7ebae7bb5b80dfd7e6)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:49:04.543801 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d933793d-2220-4a77-b510-7ebf213919f6 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:49:04.592717 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-d933793d-2220-4a77-b510-7ebf213919f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2235317fe8d4d7ebae7bb5b80dfd7e6'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:49:04.599761 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-d933793d-2220-4a77-b510-7ebf213919f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2235317fe8d4d7ebae7bb5b80dfd7e6'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:49:04.600441 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-d933793d-2220-4a77-b510-7ebf213919f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2235317fe8d4d7ebae7bb5b80dfd7e6'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:49:04.601076 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-d933793d-2220-4a77-b510-7ebf213919f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2235317fe8d4d7ebae7bb5b80dfd7e6'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:49:04.613587 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-d933793d-2220-4a77-b510-7ebf213919f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2235317fe8d4d7ebae7bb5b80dfd7e6'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:49:04.626847 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1392/2774] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Aug 6 01:49:04 2022] DELETE /identity/v3/users/d2235317fe8d4d7ebae7bb5b80dfd7e6 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:49:04.801761 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ccc537-8fa2-40f1-af54-f72d8d5e7e3c service placement] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:49:04.812820 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ccc537-8fa2-40f1-af54-f72d8d5e7e3c service placement] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:49:04.870616 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c68ef65-ec38-4e6d-9646-c88636c69be2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:49:04.871253 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c68ef65-ec38-4e6d-9646-c88636c69be2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:49:04.871538 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c68ef65-ec38-4e6d-9646-c88636c69be2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:49:04.871814 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c68ef65-ec38-4e6d-9646-c88636c69be2 admin admin] PATH_INFO: `/v3/projects/7906e11569cc47778d8ed1c6b371ae4d` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:49:04.872962 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c68ef65-ec38-4e6d-9646-c88636c69be2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7906e11569cc47778d8ed1c6b371ae4d)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:49:04.875153 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c68ef65-ec38-4e6d-9646-c88636c69be2 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:49:04.921028 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-7c68ef65-ec38-4e6d-9646-c88636c69be2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7906e11569cc47778d8ed1c6b371ae4d', 'request_id': 'req-7c68ef65-ec38-4e6d-9646-c88636c69be2'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:49:04.927040 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-7c68ef65-ec38-4e6d-9646-c88636c69be2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7906e11569cc47778d8ed1c6b371ae4d', 'request_id': 'req-7c68ef65-ec38-4e6d-9646-c88636c69be2'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:49:04.932785 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-7c68ef65-ec38-4e6d-9646-c88636c69be2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7906e11569cc47778d8ed1c6b371ae4d', 'request_id': 'req-7c68ef65-ec38-4e6d-9646-c88636c69be2'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:49:04.944230 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-7c68ef65-ec38-4e6d-9646-c88636c69be2 admin admin] The token cache is being invalidate because project 7906e11569cc47778d8ed1c6b371ae4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:49:04.944519 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-7c68ef65-ec38-4e6d-9646-c88636c69be2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:49:04.945175 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-7c68ef65-ec38-4e6d-9646-c88636c69be2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:49:04.947211 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1383/2775] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Aug 6 01:49:04 2022] DELETE /identity/v3/projects/7906e11569cc47778d8ed1c6b371ae4d => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:49:04.952205 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e05fdc2-b7eb-4aa2-ba9c-5ab267107571 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:49:04.952490 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e05fdc2-b7eb-4aa2-ba9c-5ab267107571 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:49:04.952761 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e05fdc2-b7eb-4aa2-ba9c-5ab267107571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:49:04.972327 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-7e05fdc2-b7eb-4aa2-ba9c-5ab267107571 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:49:04.973479 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-7e05fdc2-b7eb-4aa2-ba9c-5ab267107571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:49:05.030812 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1393/2776] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:49:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:49:05.039118 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3448973-7307-419c-92af-79c7b1e06eec None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:49:05.039511 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3448973-7307-419c-92af-79c7b1e06eec None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:49:05.039799 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3448973-7307-419c-92af-79c7b1e06eec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:49:05.061156 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-e3448973-7307-419c-92af-79c7b1e06eec None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:49:05.062378 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-e3448973-7307-419c-92af-79c7b1e06eec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:49:05.070243 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1384/2777] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:49:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:49:05.075289 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e05fdc2-b7eb-4aa2-ba9c-5ab267107571 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:49:05.084817 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e05fdc2-b7eb-4aa2-ba9c-5ab267107571 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:49:05.090445 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89a5700-34d5-4715-a24f-47f71c540153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:49:05.091046 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89a5700-34d5-4715-a24f-47f71c540153 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:49:05.091325 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89a5700-34d5-4715-a24f-47f71c540153 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:49:05.091599 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89a5700-34d5-4715-a24f-47f71c540153 admin admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:49:05.092287 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89a5700-34d5-4715-a24f-47f71c540153 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:49:05.092591 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89a5700-34d5-4715-a24f-47f71c540153 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:49:05.094199 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89a5700-34d5-4715-a24f-47f71c540153 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:49:05.095634 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:49:05.108158 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1394/2778] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Aug 6 01:49:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:49:05.112135 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3448973-7307-419c-92af-79c7b1e06eec None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:49:05.122330 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3448973-7307-419c-92af-79c7b1e06eec None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:49:05.127359 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b221039b-46c5-47a9-944f-fcee986891bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:49:05.127973 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b221039b-46c5-47a9-944f-fcee986891bf admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:49:05.128257 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b221039b-46c5-47a9-944f-fcee986891bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:49:05.128536 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b221039b-46c5-47a9-944f-fcee986891bf admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:49:05.129283 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b221039b-46c5-47a9-944f-fcee986891bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:49:05.131180 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b221039b-46c5-47a9-944f-fcee986891bf admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:49:05.157332 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1385/2779] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Aug 6 01:49:05 2022] POST /identity/v3/projects => generated 355 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:49:05.161716 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89a5700-34d5-4715-a24f-47f71c540153 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:49:05.172694 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89a5700-34d5-4715-a24f-47f71c540153 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:49:05.229699 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189de365-eaa1-4787-a8a8-4bcb3d6d9f4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:49:05.230249 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189de365-eaa1-4787-a8a8-4bcb3d6d9f4c admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:49:05.230527 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189de365-eaa1-4787-a8a8-4bcb3d6d9f4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:49:05.230800 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189de365-eaa1-4787-a8a8-4bcb3d6d9f4c admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:49:05.231453 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189de365-eaa1-4787-a8a8-4bcb3d6d9f4c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:49:05.233376 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189de365-eaa1-4787-a8a8-4bcb3d6d9f4c admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:49:05.254040 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1395/2780] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:49:05 2022] POST /identity/v3/users => generated 278 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:49:05.258138 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b221039b-46c5-47a9-944f-fcee986891bf admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:49:05.268380 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b221039b-46c5-47a9-944f-fcee986891bf admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:49:05.273106 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bea2dd-0c54-490f-8914-1206b762d885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:49:05.274331 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bea2dd-0c54-490f-8914-1206b762d885 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:49:05.274624 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bea2dd-0c54-490f-8914-1206b762d885 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:49:05.274905 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bea2dd-0c54-490f-8914-1206b762d885 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:49:05.275599 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bea2dd-0c54-490f-8914-1206b762d885 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:49:05.277222 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bea2dd-0c54-490f-8914-1206b762d885 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:49:05.287448 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1386/2781] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:49:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:49:05.291831 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189de365-eaa1-4787-a8a8-4bcb3d6d9f4c admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:49:05.301602 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189de365-eaa1-4787-a8a8-4bcb3d6d9f4c admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:49:05.306241 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0b364e-0f9e-46af-8c15-f22c88e275d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:49:05.306843 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0b364e-0f9e-46af-8c15-f22c88e275d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:49:05.307129 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0b364e-0f9e-46af-8c15-f22c88e275d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:49:05.307410 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0b364e-0f9e-46af-8c15-f22c88e275d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:49:05.308150 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0b364e-0f9e-46af-8c15-f22c88e275d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:49:05.309774 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0b364e-0f9e-46af-8c15-f22c88e275d4 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:49:05.319997 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1396/2782] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:49:05 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:49:05.324071 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bea2dd-0c54-490f-8914-1206b762d885 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:49:05.333718 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bea2dd-0c54-490f-8914-1206b762d885 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:49:05.338842 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0870d595-56d2-42dd-9de7-bd498e9a0c1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:49:05.339488 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0870d595-56d2-42dd-9de7-bd498e9a0c1c admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:49:05.339766 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0870d595-56d2-42dd-9de7-bd498e9a0c1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:49:05.340108 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0870d595-56d2-42dd-9de7-bd498e9a0c1c admin admin] PATH_INFO: `/v3/projects/0dd61fde839f42dd8af0ebb70a88549e/users/b71970459075438391da0a706ef7a595/roles/61b3b93d37a047e4bb8369b46fe20db9` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:49:05.355731 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0870d595-56d2-42dd-9de7-bd498e9a0c1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0dd61fde839f42dd8af0ebb70a88549e, user_id=b71970459075438391da0a706ef7a595, role_id=61b3b93d37a047e4bb8369b46fe20db9)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:49:05.358475 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0870d595-56d2-42dd-9de7-bd498e9a0c1c admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:49:05.370891 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1387/2783] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:49:05 2022] PUT /identity/v3/projects/0dd61fde839f42dd8af0ebb70a88549e/users/b71970459075438391da0a706ef7a595/roles/61b3b93d37a047e4bb8369b46fe20db9 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:49:05.382149 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd420574-51c6-47b0-8b8f-a9de9540181f None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:49:05.382440 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd420574-51c6-47b0-8b8f-a9de9540181f None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:49:05.382722 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd420574-51c6-47b0-8b8f-a9de9540181f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:49:05.411071 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-cd420574-51c6-47b0-8b8f-a9de9540181f None None] MFA Rules not processed for user `b71970459075438391da0a706ef7a595`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:49:05.412291 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-cd420574-51c6-47b0-8b8f-a9de9540181f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:49:05.475839 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1397/2784] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:49:05 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:49:05.481338 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0870d595-56d2-42dd-9de7-bd498e9a0c1c admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:49:05.494410 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:49:05.544985 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0870d595-56d2-42dd-9de7-bd498e9a0c1c admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:49:05.549714 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb3f6c7-8c23-4310-9685-9ac619a70fdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:49:05.550337 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb3f6c7-8c23-4310-9685-9ac619a70fdc admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:49:05.550628 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb3f6c7-8c23-4310-9685-9ac619a70fdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:49:05.550908 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb3f6c7-8c23-4310-9685-9ac619a70fdc admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:49:05.551583 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb3f6c7-8c23-4310-9685-9ac619a70fdc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:49:05.553539 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb3f6c7-8c23-4310-9685-9ac619a70fdc admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:49:05.581300 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1388/2785] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Aug 6 01:49:05 2022] POST /identity/v3/projects => generated 355 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:49:05.585026 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd420574-51c6-47b0-8b8f-a9de9540181f None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:49:05.596888 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd420574-51c6-47b0-8b8f-a9de9540181f None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:49:05.653162 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f6bb43-ac7b-44da-8085-2bc60fc4241f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:49:05.653774 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f6bb43-ac7b-44da-8085-2bc60fc4241f admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:49:05.654054 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f6bb43-ac7b-44da-8085-2bc60fc4241f admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:49:05.654326 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f6bb43-ac7b-44da-8085-2bc60fc4241f admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:49:05.654974 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f6bb43-ac7b-44da-8085-2bc60fc4241f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:49:05.656874 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f6bb43-ac7b-44da-8085-2bc60fc4241f admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:49:05.665753 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:49:05.680139 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1398/2786] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:49:05 2022] POST /identity/v3/users => generated 284 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:49:05.684030 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb3f6c7-8c23-4310-9685-9ac619a70fdc admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:49:05.694611 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb3f6c7-8c23-4310-9685-9ac619a70fdc admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:49:05.703288 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf53068-0eab-4b9b-9421-1182c802b6a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:49:05.703921 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf53068-0eab-4b9b-9421-1182c802b6a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:49:05.704210 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf53068-0eab-4b9b-9421-1182c802b6a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:49:05.704491 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf53068-0eab-4b9b-9421-1182c802b6a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:49:05.705219 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf53068-0eab-4b9b-9421-1182c802b6a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:49:05.706831 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf53068-0eab-4b9b-9421-1182c802b6a9 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:49:05.719012 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1389/2787] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:49:05 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 01:49:05.723169 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f6bb43-ac7b-44da-8085-2bc60fc4241f admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:49:05.734003 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f6bb43-ac7b-44da-8085-2bc60fc4241f admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:49:05.738485 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c5b7db-50b9-4105-8b6a-3f84194400af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:49:05.739159 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c5b7db-50b9-4105-8b6a-3f84194400af admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:49:05.739418 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c5b7db-50b9-4105-8b6a-3f84194400af admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:49:05.739691 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c5b7db-50b9-4105-8b6a-3f84194400af admin admin] PATH_INFO: `/v3/projects/c1becc466b10483484e9bd16000af332/users/bf29aaf741af40a8bf270972c872f655/roles/4559b9a0837249f895a13c70096f8d38` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:49:05.756253 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c5b7db-50b9-4105-8b6a-3f84194400af admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1becc466b10483484e9bd16000af332, user_id=bf29aaf741af40a8bf270972c872f655, role_id=4559b9a0837249f895a13c70096f8d38)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:49:05.759922 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c5b7db-50b9-4105-8b6a-3f84194400af admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:49:05.774631 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1399/2788] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:49:05 2022] PUT /identity/v3/projects/c1becc466b10483484e9bd16000af332/users/bf29aaf741af40a8bf270972c872f655/roles/4559b9a0837249f895a13c70096f8d38 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:49:05.778544 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf53068-0eab-4b9b-9421-1182c802b6a9 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:49:05.788865 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf53068-0eab-4b9b-9421-1182c802b6a9 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:49:05.845429 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa7301b-85f9-45ce-b761-b2dad3f7ef50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:49:05.846043 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa7301b-85f9-45ce-b761-b2dad3f7ef50 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:49:05.846336 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa7301b-85f9-45ce-b761-b2dad3f7ef50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:49:05.846594 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa7301b-85f9-45ce-b761-b2dad3f7ef50 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:49:05.847271 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa7301b-85f9-45ce-b761-b2dad3f7ef50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:49:05.848850 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa7301b-85f9-45ce-b761-b2dad3f7ef50 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:49:05.858763 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1390/2789] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:49:05 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 01:49:05.862883 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c5b7db-50b9-4105-8b6a-3f84194400af admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:49:05.873723 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c5b7db-50b9-4105-8b6a-3f84194400af admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:49:05.878437 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d22a34-82f8-4405-9611-d3be1489188c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:49:05.879126 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d22a34-82f8-4405-9611-d3be1489188c admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:49:05.879407 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d22a34-82f8-4405-9611-d3be1489188c admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:49:05.879685 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d22a34-82f8-4405-9611-d3be1489188c admin admin] PATH_INFO: `/v3/projects/c1becc466b10483484e9bd16000af332/users/bf29aaf741af40a8bf270972c872f655/roles/4559b9a0837249f895a13c70096f8d38` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:49:05.881696 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d22a34-82f8-4405-9611-d3be1489188c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1becc466b10483484e9bd16000af332, user_id=bf29aaf741af40a8bf270972c872f655, role_id=4559b9a0837249f895a13c70096f8d38)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:49:05.884988 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d22a34-82f8-4405-9611-d3be1489188c admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:49:05.895155 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1400/2790] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:49:05 2022] PUT /identity/v3/projects/c1becc466b10483484e9bd16000af332/users/bf29aaf741af40a8bf270972c872f655/roles/4559b9a0837249f895a13c70096f8d38 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:49:05.906067 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09398e8a-b469-43b3-be63-9f4ba4e95ccc None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:49:05.906356 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09398e8a-b469-43b3-be63-9f4ba4e95ccc None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:49:05.906634 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09398e8a-b469-43b3-be63-9f4ba4e95ccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:49:05.933963 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-09398e8a-b469-43b3-be63-9f4ba4e95ccc None None] MFA Rules not processed for user `bf29aaf741af40a8bf270972c872f655`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:49:05.935158 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-09398e8a-b469-43b3-be63-9f4ba4e95ccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:49:06.002170 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1391/2791] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:49:05 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:49:06.014160 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d22a34-82f8-4405-9611-d3be1489188c admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:49:06.015877 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-36d22a34-82f8-4405-9611-d3be1489188c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:49:06.073984 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d22a34-82f8-4405-9611-d3be1489188c admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:49:06.077539 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffe605e-5f6a-4b73-aabd-79662615249b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1be1e6173e374a60b9e1fbf51a1ab571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:49:06.078140 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffe605e-5f6a-4b73-aabd-79662615249b service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:49:06.078416 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffe605e-5f6a-4b73-aabd-79662615249b service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:49:06.078687 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffe605e-5f6a-4b73-aabd-79662615249b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:49:06.087653 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffe605e-5f6a-4b73-aabd-79662615249b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:49:06.089109 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffe605e-5f6a-4b73-aabd-79662615249b service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:49:06.098212 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1401/2792] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:49:06 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:49:06.174073 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09398e8a-b469-43b3-be63-9f4ba4e95ccc None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:49:06.184295 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09398e8a-b469-43b3-be63-9f4ba4e95ccc None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:49:06.187877 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd8fe67-1c70-4cbd-bd82-4e5e7598808b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1be1e6173e374a60b9e1fbf51a1ab571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:49:06.188483 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd8fe67-1c70-4cbd-bd82-4e5e7598808b service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:49:06.188761 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd8fe67-1c70-4cbd-bd82-4e5e7598808b service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:49:06.189032 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd8fe67-1c70-4cbd-bd82-4e5e7598808b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:49:06.194377 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:49:06.249427 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd8fe67-1c70-4cbd-bd82-4e5e7598808b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:49:06.250791 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd8fe67-1c70-4cbd-bd82-4e5e7598808b service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:49:06.260378 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1392/2793] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:49:06 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:49:06.972664 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffe605e-5f6a-4b73-aabd-79662615249b service cinder] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:49:06.974211 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-1ffe605e-5f6a-4b73-aabd-79662615249b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:49:07.030717 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffe605e-5f6a-4b73-aabd-79662615249b service cinder] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:49:07.034317 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d91190-566b-46e3-ac54-5e8aeff36fca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:49:07.034926 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d91190-566b-46e3-ac54-5e8aeff36fca service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:49:07.035209 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d91190-566b-46e3-ac54-5e8aeff36fca service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:49:07.035483 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d91190-566b-46e3-ac54-5e8aeff36fca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:49:07.037246 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-47d91190-566b-46e3-ac54-5e8aeff36fca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:49:07.047591 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d91190-566b-46e3-ac54-5e8aeff36fca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:49:07.049027 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d91190-566b-46e3-ac54-5e8aeff36fca service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:49:07.057027 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1402/2794] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:49:06 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:49:16.255671 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=24 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:49:17.053473 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:INSERT=4,keystone:SELECT=69 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:49:21.175718 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd8fe67-1c70-4cbd-bd82-4e5e7598808b service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:49:21.238181 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd8fe67-1c70-4cbd-bd82-4e5e7598808b service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:49:21.242906 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453c5ca0-c701-4e82-8abf-ce5fe58db9c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:49:21.243541 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453c5ca0-c701-4e82-8abf-ce5fe58db9c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:49:21.243824 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453c5ca0-c701-4e82-8abf-ce5fe58db9c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:49:21.244103 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453c5ca0-c701-4e82-8abf-ce5fe58db9c3 admin admin] PATH_INFO: `/v3/users/b71970459075438391da0a706ef7a595` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:49:21.245302 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453c5ca0-c701-4e82-8abf-ce5fe58db9c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b71970459075438391da0a706ef7a595)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:49:21.247282 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453c5ca0-c701-4e82-8abf-ce5fe58db9c3 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:49:21.293056 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-453c5ca0-c701-4e82-8abf-ce5fe58db9c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b71970459075438391da0a706ef7a595'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:49:21.293720 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-453c5ca0-c701-4e82-8abf-ce5fe58db9c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b71970459075438391da0a706ef7a595'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:49:21.294317 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-453c5ca0-c701-4e82-8abf-ce5fe58db9c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b71970459075438391da0a706ef7a595'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:49:21.306004 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-453c5ca0-c701-4e82-8abf-ce5fe58db9c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b71970459075438391da0a706ef7a595'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:49:21.312628 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-453c5ca0-c701-4e82-8abf-ce5fe58db9c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b71970459075438391da0a706ef7a595'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:49:21.327456 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1393/2795] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Aug 6 01:49:21 2022] DELETE /identity/v3/users/b71970459075438391da0a706ef7a595 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:49:21.331836 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa75356-ebaf-458f-8702-8bc27d428f63 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:49:21.332346 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa75356-ebaf-458f-8702-8bc27d428f63 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:49:21.332462 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa75356-ebaf-458f-8702-8bc27d428f63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:49:21.351882 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-dfa75356-ebaf-458f-8702-8bc27d428f63 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:49:21.353077 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-dfa75356-ebaf-458f-8702-8bc27d428f63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:49:21.409990 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1403/2796] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:49:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:49:21.419715 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453c5ca0-c701-4e82-8abf-ce5fe58db9c3 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:49:21.421121 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-453c5ca0-c701-4e82-8abf-ce5fe58db9c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:49:21.477695 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453c5ca0-c701-4e82-8abf-ce5fe58db9c3 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:49:21.481404 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d7a9ee-5fa1-4dd9-91b9-1c2eb91eab4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:49:21.482014 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d7a9ee-5fa1-4dd9-91b9-1c2eb91eab4b service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:49:21.482297 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d7a9ee-5fa1-4dd9-91b9-1c2eb91eab4b service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:49:21.482574 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d7a9ee-5fa1-4dd9-91b9-1c2eb91eab4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:49:21.491544 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d7a9ee-5fa1-4dd9-91b9-1c2eb91eab4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:49:21.493207 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d7a9ee-5fa1-4dd9-91b9-1c2eb91eab4b service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:49:21.502739 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1394/2797] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:49:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:49:21.770745 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa75356-ebaf-458f-8702-8bc27d428f63 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:49:21.772262 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-dfa75356-ebaf-458f-8702-8bc27d428f63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:49:21.785217 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa75356-ebaf-458f-8702-8bc27d428f63 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:49:21.790597 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b78a106-2301-4dc9-bbba-2c0d597f0f41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:49:21.791243 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b78a106-2301-4dc9-bbba-2c0d597f0f41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:49:21.791520 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b78a106-2301-4dc9-bbba-2c0d597f0f41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:49:21.791790 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b78a106-2301-4dc9-bbba-2c0d597f0f41 admin admin] PATH_INFO: `/v3/projects/0dd61fde839f42dd8af0ebb70a88549e` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:49:21.792937 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b78a106-2301-4dc9-bbba-2c0d597f0f41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0dd61fde839f42dd8af0ebb70a88549e)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:49:21.794912 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b78a106-2301-4dc9-bbba-2c0d597f0f41 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:49:21.842575 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-4b78a106-2301-4dc9-bbba-2c0d597f0f41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0dd61fde839f42dd8af0ebb70a88549e', 'request_id': 'req-4b78a106-2301-4dc9-bbba-2c0d597f0f41'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:49:21.848800 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-4b78a106-2301-4dc9-bbba-2c0d597f0f41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0dd61fde839f42dd8af0ebb70a88549e', 'request_id': 'req-4b78a106-2301-4dc9-bbba-2c0d597f0f41'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:49:21.855078 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-4b78a106-2301-4dc9-bbba-2c0d597f0f41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0dd61fde839f42dd8af0ebb70a88549e', 'request_id': 'req-4b78a106-2301-4dc9-bbba-2c0d597f0f41'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:49:21.866768 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-4b78a106-2301-4dc9-bbba-2c0d597f0f41 admin admin] The token cache is being invalidate because project 0dd61fde839f42dd8af0ebb70a88549e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:49:21.867095 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-4b78a106-2301-4dc9-bbba-2c0d597f0f41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:49:21.867811 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-4b78a106-2301-4dc9-bbba-2c0d597f0f41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:49:21.869338 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1404/2798] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Aug 6 01:49:21 2022] DELETE /identity/v3/projects/0dd61fde839f42dd8af0ebb70a88549e => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:49:21.873648 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d7a9ee-5fa1-4dd9-91b9-1c2eb91eab4b service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:49:21.883235 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d7a9ee-5fa1-4dd9-91b9-1c2eb91eab4b service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:49:21.901559 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:49:21.944122 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4b91ec-1975-4385-9b9e-ba794e3ef110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:49:21.944790 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4b91ec-1975-4385-9b9e-ba794e3ef110 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:49:21.945093 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4b91ec-1975-4385-9b9e-ba794e3ef110 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:49:21.945469 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4b91ec-1975-4385-9b9e-ba794e3ef110 admin admin] PATH_INFO: `/v3/users/bf29aaf741af40a8bf270972c872f655` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:49:21.946675 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4b91ec-1975-4385-9b9e-ba794e3ef110 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf29aaf741af40a8bf270972c872f655)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:49:21.948650 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4b91ec-1975-4385-9b9e-ba794e3ef110 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:49:21.995365 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-fe4b91ec-1975-4385-9b9e-ba794e3ef110 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf29aaf741af40a8bf270972c872f655'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:49:21.996029 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-fe4b91ec-1975-4385-9b9e-ba794e3ef110 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf29aaf741af40a8bf270972c872f655'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:49:21.996640 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-fe4b91ec-1975-4385-9b9e-ba794e3ef110 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf29aaf741af40a8bf270972c872f655'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:49:22.008756 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-fe4b91ec-1975-4385-9b9e-ba794e3ef110 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf29aaf741af40a8bf270972c872f655'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:49:22.015485 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-fe4b91ec-1975-4385-9b9e-ba794e3ef110 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf29aaf741af40a8bf270972c872f655'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:49:22.027690 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1395/2799] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Aug 6 01:49:21 2022] DELETE /identity/v3/users/bf29aaf741af40a8bf270972c872f655 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:49:22.301175 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b78a106-2301-4dc9-bbba-2c0d597f0f41 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:49:22.302692 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-4b78a106-2301-4dc9-bbba-2c0d597f0f41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:49:22.316011 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b78a106-2301-4dc9-bbba-2c0d597f0f41 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:49:22.374924 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f6b4d1-2e71-4358-9b06-649006bca302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:49:22.375654 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f6b4d1-2e71-4358-9b06-649006bca302 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:49:22.375838 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f6b4d1-2e71-4358-9b06-649006bca302 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:49:22.376114 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f6b4d1-2e71-4358-9b06-649006bca302 admin admin] PATH_INFO: `/v3/projects/c1becc466b10483484e9bd16000af332` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:49:22.377312 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f6b4d1-2e71-4358-9b06-649006bca302 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1becc466b10483484e9bd16000af332)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:49:22.379222 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f6b4d1-2e71-4358-9b06-649006bca302 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:49:22.411154 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:49:22.428215 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-39f6b4d1-2e71-4358-9b06-649006bca302 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1becc466b10483484e9bd16000af332', 'request_id': 'req-39f6b4d1-2e71-4358-9b06-649006bca302'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:49:22.435050 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-39f6b4d1-2e71-4358-9b06-649006bca302 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1becc466b10483484e9bd16000af332', 'request_id': 'req-39f6b4d1-2e71-4358-9b06-649006bca302'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:49:22.440489 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-39f6b4d1-2e71-4358-9b06-649006bca302 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1becc466b10483484e9bd16000af332', 'request_id': 'req-39f6b4d1-2e71-4358-9b06-649006bca302'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:49:22.453586 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-39f6b4d1-2e71-4358-9b06-649006bca302 admin admin] The token cache is being invalidate because project c1becc466b10483484e9bd16000af332 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:49:22.453875 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-39f6b4d1-2e71-4358-9b06-649006bca302 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:49:22.454491 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-39f6b4d1-2e71-4358-9b06-649006bca302 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:49:22.455968 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1405/2800] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Aug 6 01:49:22 2022] DELETE /identity/v3/projects/c1becc466b10483484e9bd16000af332 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:49:22.461125 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87207a5f-060a-403f-a968-03ea5150a67f None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:49:22.461713 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87207a5f-060a-403f-a968-03ea5150a67f None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:49:22.461713 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87207a5f-060a-403f-a968-03ea5150a67f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:49:22.481119 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-87207a5f-060a-403f-a968-03ea5150a67f None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:49:22.482256 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-87207a5f-060a-403f-a968-03ea5150a67f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:49:22.540802 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1396/2801] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:49:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:49:22.550070 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cffd3fc-3ca6-4f63-b397-9070e4e4b858 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:49:22.550363 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cffd3fc-3ca6-4f63-b397-9070e4e4b858 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:49:22.550647 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cffd3fc-3ca6-4f63-b397-9070e4e4b858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:49:22.571189 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-0cffd3fc-3ca6-4f63-b397-9070e4e4b858 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:49:22.572421 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-0cffd3fc-3ca6-4f63-b397-9070e4e4b858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:49:22.580362 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1406/2802] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:49:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:49:22.584585 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87207a5f-060a-403f-a968-03ea5150a67f None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:49:22.594555 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87207a5f-060a-403f-a968-03ea5150a67f None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:49:22.599244 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197c1c6c-e9d6-490e-823a-c6d59f44fc4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:49:22.599866 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197c1c6c-e9d6-490e-823a-c6d59f44fc4b admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:49:22.600149 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197c1c6c-e9d6-490e-823a-c6d59f44fc4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:49:22.600429 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197c1c6c-e9d6-490e-823a-c6d59f44fc4b admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:49:22.601155 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197c1c6c-e9d6-490e-823a-c6d59f44fc4b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:49:22.601470 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197c1c6c-e9d6-490e-823a-c6d59f44fc4b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:49:22.603063 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197c1c6c-e9d6-490e-823a-c6d59f44fc4b admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:49:22.614525 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1397/2803] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Aug 6 01:49:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:49:22.619516 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cffd3fc-3ca6-4f63-b397-9070e4e4b858 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:49:22.628871 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cffd3fc-3ca6-4f63-b397-9070e4e4b858 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:49:22.634129 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f140088c-608e-43ff-836e-3ad0ddfe1dd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:49:22.634735 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f140088c-608e-43ff-836e-3ad0ddfe1dd1 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:49:22.635021 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f140088c-608e-43ff-836e-3ad0ddfe1dd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:49:22.635296 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f140088c-608e-43ff-836e-3ad0ddfe1dd1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:49:22.635964 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f140088c-608e-43ff-836e-3ad0ddfe1dd1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:49:22.637903 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f140088c-608e-43ff-836e-3ad0ddfe1dd1 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:49:22.664180 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1407/2804] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Aug 6 01:49:22 2022] POST /identity/v3/projects => generated 367 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:49:22.668370 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197c1c6c-e9d6-490e-823a-c6d59f44fc4b admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:49:22.678307 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197c1c6c-e9d6-490e-823a-c6d59f44fc4b admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:49:22.704624 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:49:22.736525 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1602ead-5fc9-454c-bb4f-1d257238b62b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:49:22.737127 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1602ead-5fc9-454c-bb4f-1d257238b62b admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:49:22.737410 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1602ead-5fc9-454c-bb4f-1d257238b62b admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:49:22.737682 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1602ead-5fc9-454c-bb4f-1d257238b62b admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:49:22.738341 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1602ead-5fc9-454c-bb4f-1d257238b62b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:49:22.740238 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1602ead-5fc9-454c-bb4f-1d257238b62b admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:49:22.762364 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1398/2805] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:49:22 2022] POST /identity/v3/users => generated 284 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:49:22.766524 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f140088c-608e-43ff-836e-3ad0ddfe1dd1 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:49:22.776698 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f140088c-608e-43ff-836e-3ad0ddfe1dd1 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:49:22.781793 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4349379d-1581-4408-84d1-eb87ff2727ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:49:22.782405 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4349379d-1581-4408-84d1-eb87ff2727ee admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:49:22.782693 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4349379d-1581-4408-84d1-eb87ff2727ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:49:22.782969 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4349379d-1581-4408-84d1-eb87ff2727ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:49:22.783672 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4349379d-1581-4408-84d1-eb87ff2727ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:49:22.785328 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4349379d-1581-4408-84d1-eb87ff2727ee admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:49:22.795920 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1408/2806] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:49:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 01:49:22.799898 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1602ead-5fc9-454c-bb4f-1d257238b62b admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:49:22.810307 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1602ead-5fc9-454c-bb4f-1d257238b62b admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:49:22.814850 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0b41f2-a6d0-4685-b4a2-fe45a2d564b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:49:22.815452 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0b41f2-a6d0-4685-b4a2-fe45a2d564b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:49:22.815733 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0b41f2-a6d0-4685-b4a2-fe45a2d564b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:49:22.816010 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0b41f2-a6d0-4685-b4a2-fe45a2d564b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:49:22.816684 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0b41f2-a6d0-4685-b4a2-fe45a2d564b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:49:22.818325 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0b41f2-a6d0-4685-b4a2-fe45a2d564b5 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:49:22.828726 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1399/2807] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:49:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:49:22.832694 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4349379d-1581-4408-84d1-eb87ff2727ee admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:49:22.842877 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4349379d-1581-4408-84d1-eb87ff2727ee admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:49:22.847519 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b42a69e-acdb-42c5-ad6b-0131080f16f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:49:22.848388 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b42a69e-acdb-42c5-ad6b-0131080f16f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:49:22.848716 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b42a69e-acdb-42c5-ad6b-0131080f16f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:49:22.849214 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b42a69e-acdb-42c5-ad6b-0131080f16f1 admin admin] PATH_INFO: `/v3/projects/c8cb043953aa46dea88e909d9a8e1197/users/04df337816cf4679af5b5e82ac8fe782/roles/61b3b93d37a047e4bb8369b46fe20db9` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:49:22.864362 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b42a69e-acdb-42c5-ad6b-0131080f16f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8cb043953aa46dea88e909d9a8e1197, user_id=04df337816cf4679af5b5e82ac8fe782, role_id=61b3b93d37a047e4bb8369b46fe20db9)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:49:22.867589 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b42a69e-acdb-42c5-ad6b-0131080f16f1 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:49:22.880631 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1409/2808] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:49:22 2022] PUT /identity/v3/projects/c8cb043953aa46dea88e909d9a8e1197/users/04df337816cf4679af5b5e82ac8fe782/roles/61b3b93d37a047e4bb8369b46fe20db9 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:49:22.890845 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a73436a-1e60-46b9-9d7e-4449e6ca26a7 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:49:22.891161 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a73436a-1e60-46b9-9d7e-4449e6ca26a7 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:49:22.891466 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a73436a-1e60-46b9-9d7e-4449e6ca26a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:49:22.919162 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-0a73436a-1e60-46b9-9d7e-4449e6ca26a7 None None] MFA Rules not processed for user `04df337816cf4679af5b5e82ac8fe782`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:49:22.920412 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-0a73436a-1e60-46b9-9d7e-4449e6ca26a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:49:22.985824 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1400/2809] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:49:22 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:49:22.990200 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b42a69e-acdb-42c5-ad6b-0131080f16f1 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:49:23.052663 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b42a69e-acdb-42c5-ad6b-0131080f16f1 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:49:23.057561 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82643f9c-1701-4977-8014-d7b9ab90c768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:49:23.058173 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82643f9c-1701-4977-8014-d7b9ab90c768 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:49:23.058455 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82643f9c-1701-4977-8014-d7b9ab90c768 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:49:23.058728 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82643f9c-1701-4977-8014-d7b9ab90c768 admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:49:23.059399 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82643f9c-1701-4977-8014-d7b9ab90c768 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:49:23.061313 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82643f9c-1701-4977-8014-d7b9ab90c768 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:49:23.088008 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1410/2810] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Aug 6 01:49:22 2022] POST /identity/v3/projects => generated 369 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:49:23.092025 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a73436a-1e60-46b9-9d7e-4449e6ca26a7 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:49:23.103204 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a73436a-1e60-46b9-9d7e-4449e6ca26a7 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:49:23.159458 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651ed219-54eb-4479-bd9d-a6c5dcac8114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:49:23.160070 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651ed219-54eb-4479-bd9d-a6c5dcac8114 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:49:23.160348 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651ed219-54eb-4479-bd9d-a6c5dcac8114 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:49:23.160620 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651ed219-54eb-4479-bd9d-a6c5dcac8114 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:49:23.161320 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651ed219-54eb-4479-bd9d-a6c5dcac8114 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:49:23.163225 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651ed219-54eb-4479-bd9d-a6c5dcac8114 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:49:23.184025 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1401/2811] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:49:23 2022] POST /identity/v3/users => generated 291 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:49:23.187981 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82643f9c-1701-4977-8014-d7b9ab90c768 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:49:23.199140 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82643f9c-1701-4977-8014-d7b9ab90c768 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:49:23.203727 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7a7270-e8c5-4b75-b0b0-1758d4868043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:49:23.204382 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7a7270-e8c5-4b75-b0b0-1758d4868043 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:49:23.204682 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7a7270-e8c5-4b75-b0b0-1758d4868043 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:49:23.204958 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7a7270-e8c5-4b75-b0b0-1758d4868043 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:49:23.205679 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7a7270-e8c5-4b75-b0b0-1758d4868043 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:49:23.207281 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7a7270-e8c5-4b75-b0b0-1758d4868043 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:49:23.218524 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1411/2812] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:49:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:49:23.222600 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651ed219-54eb-4479-bd9d-a6c5dcac8114 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:49:23.233809 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651ed219-54eb-4479-bd9d-a6c5dcac8114 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:49:23.238520 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b5d9a7-370e-465d-acdf-e9186fa8e918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:49:23.239172 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b5d9a7-370e-465d-acdf-e9186fa8e918 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:49:23.239452 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b5d9a7-370e-465d-acdf-e9186fa8e918 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:49:23.239730 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b5d9a7-370e-465d-acdf-e9186fa8e918 admin admin] PATH_INFO: `/v3/projects/01c9a2e469214b2385f22d776e724021/users/468ebf7438f74c39b274b522cd8de3b1/roles/4559b9a0837249f895a13c70096f8d38` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:49:23.255260 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b5d9a7-370e-465d-acdf-e9186fa8e918 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01c9a2e469214b2385f22d776e724021, user_id=468ebf7438f74c39b274b522cd8de3b1, role_id=4559b9a0837249f895a13c70096f8d38)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:49:23.258628 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b5d9a7-370e-465d-acdf-e9186fa8e918 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:49:23.270776 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1402/2813] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:49:23 2022] PUT /identity/v3/projects/01c9a2e469214b2385f22d776e724021/users/468ebf7438f74c39b274b522cd8de3b1/roles/4559b9a0837249f895a13c70096f8d38 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:49:23.274968 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7a7270-e8c5-4b75-b0b0-1758d4868043 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:49:23.284734 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7a7270-e8c5-4b75-b0b0-1758d4868043 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:49:23.303850 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:49:23.342444 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d21e1ce-1924-4389-88d8-c7e8154e2dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:49:23.343060 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d21e1ce-1924-4389-88d8-c7e8154e2dcb admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:49:23.343341 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d21e1ce-1924-4389-88d8-c7e8154e2dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:49:23.343619 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d21e1ce-1924-4389-88d8-c7e8154e2dcb admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:49:23.344311 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d21e1ce-1924-4389-88d8-c7e8154e2dcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:49:23.346090 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d21e1ce-1924-4389-88d8-c7e8154e2dcb admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:49:23.355807 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1412/2814] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:49:23 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:49:23.360137 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b5d9a7-370e-465d-acdf-e9186fa8e918 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:49:23.365639 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:49:23.372433 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b5d9a7-370e-465d-acdf-e9186fa8e918 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:49:23.377614 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26bf156-5dae-4f89-87c2-5eb434446851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:49:23.378278 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26bf156-5dae-4f89-87c2-5eb434446851 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:49:23.378565 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26bf156-5dae-4f89-87c2-5eb434446851 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:49:23.378846 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26bf156-5dae-4f89-87c2-5eb434446851 admin admin] PATH_INFO: `/v3/projects/01c9a2e469214b2385f22d776e724021/users/468ebf7438f74c39b274b522cd8de3b1/roles/4559b9a0837249f895a13c70096f8d38` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:49:23.380818 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26bf156-5dae-4f89-87c2-5eb434446851 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01c9a2e469214b2385f22d776e724021, user_id=468ebf7438f74c39b274b522cd8de3b1, role_id=4559b9a0837249f895a13c70096f8d38)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:49:23.384289 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26bf156-5dae-4f89-87c2-5eb434446851 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:49:23.394637 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1403/2815] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:49:23 2022] PUT /identity/v3/projects/01c9a2e469214b2385f22d776e724021/users/468ebf7438f74c39b274b522cd8de3b1/roles/4559b9a0837249f895a13c70096f8d38 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:49:23.406219 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d063e03-354b-4d94-ab9b-39640fbcdf6b None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:49:23.406835 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d063e03-354b-4d94-ab9b-39640fbcdf6b None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:49:23.406928 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d063e03-354b-4d94-ab9b-39640fbcdf6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:49:23.436508 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-7d063e03-354b-4d94-ab9b-39640fbcdf6b None None] MFA Rules not processed for user `468ebf7438f74c39b274b522cd8de3b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:49:23.437783 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-7d063e03-354b-4d94-ab9b-39640fbcdf6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:49:23.505415 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1413/2816] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:49:23 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:49:23.516610 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26bf156-5dae-4f89-87c2-5eb434446851 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:49:23.518401 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-b26bf156-5dae-4f89-87c2-5eb434446851 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:49:23.574766 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26bf156-5dae-4f89-87c2-5eb434446851 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:49:23.578361 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e47f1d-4aae-4721-b1a9-c13dcbf67e53 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1be1e6173e374a60b9e1fbf51a1ab571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:49:23.578966 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e47f1d-4aae-4721-b1a9-c13dcbf67e53 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:49:23.579243 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e47f1d-4aae-4721-b1a9-c13dcbf67e53 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:49:23.579515 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e47f1d-4aae-4721-b1a9-c13dcbf67e53 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:49:23.588627 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e47f1d-4aae-4721-b1a9-c13dcbf67e53 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:49:23.590107 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e47f1d-4aae-4721-b1a9-c13dcbf67e53 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:49:23.599473 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1404/2817] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:49:23 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:49:24.979233 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d063e03-354b-4d94-ab9b-39640fbcdf6b None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:49:25.041720 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d063e03-354b-4d94-ab9b-39640fbcdf6b None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:49:25.046251 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36e961e-569c-4fbf-a034-55a2942e20be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:49:25.046802 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36e961e-569c-4fbf-a034-55a2942e20be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:49:25.047078 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36e961e-569c-4fbf-a034-55a2942e20be admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:49:25.047348 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36e961e-569c-4fbf-a034-55a2942e20be admin admin] PATH_INFO: `/v3/users/04df337816cf4679af5b5e82ac8fe782` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:49:25.048545 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36e961e-569c-4fbf-a034-55a2942e20be admin admin] RBAC: Authorizing `identity:delete_user(user_id=04df337816cf4679af5b5e82ac8fe782)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:49:25.050617 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36e961e-569c-4fbf-a034-55a2942e20be admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:49:25.097305 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-c36e961e-569c-4fbf-a034-55a2942e20be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04df337816cf4679af5b5e82ac8fe782'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:49:25.098482 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:49:25.106741 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-c36e961e-569c-4fbf-a034-55a2942e20be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04df337816cf4679af5b5e82ac8fe782'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:49:25.107393 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-c36e961e-569c-4fbf-a034-55a2942e20be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04df337816cf4679af5b5e82ac8fe782'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:49:25.108026 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-c36e961e-569c-4fbf-a034-55a2942e20be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04df337816cf4679af5b5e82ac8fe782'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:49:25.119954 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-c36e961e-569c-4fbf-a034-55a2942e20be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04df337816cf4679af5b5e82ac8fe782'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:49:25.137366 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1414/2818] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Aug 6 01:49:24 2022] DELETE /identity/v3/users/04df337816cf4679af5b5e82ac8fe782 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:49:25.141943 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3fb796-2d38-491c-a29c-6cd85c72dd84 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:49:25.142436 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3fb796-2d38-491c-a29c-6cd85c72dd84 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:49:25.142543 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3fb796-2d38-491c-a29c-6cd85c72dd84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:49:25.163564 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-4d3fb796-2d38-491c-a29c-6cd85c72dd84 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:49:25.164792 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-4d3fb796-2d38-491c-a29c-6cd85c72dd84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:49:25.223211 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1405/2819] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:49:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:49:25.234306 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36e961e-569c-4fbf-a034-55a2942e20be admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:49:25.235709 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-c36e961e-569c-4fbf-a034-55a2942e20be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:49:25.292853 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36e961e-569c-4fbf-a034-55a2942e20be admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:49:25.296537 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb556818-1337-4d23-bb9e-35616d426908 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:49:25.297115 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb556818-1337-4d23-bb9e-35616d426908 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:49:25.297392 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb556818-1337-4d23-bb9e-35616d426908 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:49:25.297659 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb556818-1337-4d23-bb9e-35616d426908 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:49:25.307141 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb556818-1337-4d23-bb9e-35616d426908 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:49:25.308581 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb556818-1337-4d23-bb9e-35616d426908 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:49:25.317988 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1415/2820] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:49:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:49:25.580847 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3fb796-2d38-491c-a29c-6cd85c72dd84 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:49:25.582351 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-4d3fb796-2d38-491c-a29c-6cd85c72dd84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:49:25.596667 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3fb796-2d38-491c-a29c-6cd85c72dd84 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:49:25.601892 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa0c3d3-8492-42cb-8c40-82f2194aeace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:49:25.603322 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa0c3d3-8492-42cb-8c40-82f2194aeace admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:49:25.603785 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa0c3d3-8492-42cb-8c40-82f2194aeace admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:49:25.603911 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa0c3d3-8492-42cb-8c40-82f2194aeace admin admin] PATH_INFO: `/v3/projects/c8cb043953aa46dea88e909d9a8e1197` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:49:25.605071 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa0c3d3-8492-42cb-8c40-82f2194aeace admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8cb043953aa46dea88e909d9a8e1197)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:49:25.607038 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa0c3d3-8492-42cb-8c40-82f2194aeace admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:49:25.654506 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-8aa0c3d3-8492-42cb-8c40-82f2194aeace admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8cb043953aa46dea88e909d9a8e1197', 'request_id': 'req-8aa0c3d3-8492-42cb-8c40-82f2194aeace'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:49:25.660573 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-8aa0c3d3-8492-42cb-8c40-82f2194aeace admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8cb043953aa46dea88e909d9a8e1197', 'request_id': 'req-8aa0c3d3-8492-42cb-8c40-82f2194aeace'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:49:25.666811 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-8aa0c3d3-8492-42cb-8c40-82f2194aeace admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8cb043953aa46dea88e909d9a8e1197', 'request_id': 'req-8aa0c3d3-8492-42cb-8c40-82f2194aeace'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:49:25.679198 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-8aa0c3d3-8492-42cb-8c40-82f2194aeace admin admin] The token cache is being invalidate because project c8cb043953aa46dea88e909d9a8e1197 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:49:25.679489 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-8aa0c3d3-8492-42cb-8c40-82f2194aeace admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:49:25.680112 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-8aa0c3d3-8492-42cb-8c40-82f2194aeace admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:49:25.681698 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1406/2821] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Aug 6 01:49:25 2022] DELETE /identity/v3/projects/c8cb043953aa46dea88e909d9a8e1197 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:49:25.685717 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb556818-1337-4d23-bb9e-35616d426908 service neutron] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:49:25.695553 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb556818-1337-4d23-bb9e-35616d426908 service neutron] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:49:25.751603 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71788c9a-e993-400a-befa-a2a6ff32a6de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:49:25.752278 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71788c9a-e993-400a-befa-a2a6ff32a6de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:49:25.752565 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71788c9a-e993-400a-befa-a2a6ff32a6de admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:49:25.752845 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71788c9a-e993-400a-befa-a2a6ff32a6de admin admin] PATH_INFO: `/v3/users/468ebf7438f74c39b274b522cd8de3b1` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:49:25.754043 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71788c9a-e993-400a-befa-a2a6ff32a6de admin admin] RBAC: Authorizing `identity:delete_user(user_id=468ebf7438f74c39b274b522cd8de3b1)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:49:25.755994 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71788c9a-e993-400a-befa-a2a6ff32a6de admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:49:25.800977 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-71788c9a-e993-400a-befa-a2a6ff32a6de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '468ebf7438f74c39b274b522cd8de3b1'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:49:25.807695 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-71788c9a-e993-400a-befa-a2a6ff32a6de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '468ebf7438f74c39b274b522cd8de3b1'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:49:25.808341 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-71788c9a-e993-400a-befa-a2a6ff32a6de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '468ebf7438f74c39b274b522cd8de3b1'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:49:25.808964 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-71788c9a-e993-400a-befa-a2a6ff32a6de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '468ebf7438f74c39b274b522cd8de3b1'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:49:25.818688 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:49:25.821703 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-71788c9a-e993-400a-befa-a2a6ff32a6de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '468ebf7438f74c39b274b522cd8de3b1'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:49:25.834925 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1416/2822] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Aug 6 01:49:25 2022] DELETE /identity/v3/users/468ebf7438f74c39b274b522cd8de3b1 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:49:26.104150 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa0c3d3-8492-42cb-8c40-82f2194aeace admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:49:26.105699 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-8aa0c3d3-8492-42cb-8c40-82f2194aeace admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:49:26.118825 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa0c3d3-8492-42cb-8c40-82f2194aeace admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:49:26.130909 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:49:26.178602 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79070698-149f-4262-958c-695929768cae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:49:26.179289 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79070698-149f-4262-958c-695929768cae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:49:26.179508 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79070698-149f-4262-958c-695929768cae admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:49:26.179778 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79070698-149f-4262-958c-695929768cae admin admin] PATH_INFO: `/v3/projects/01c9a2e469214b2385f22d776e724021` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:49:26.181651 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79070698-149f-4262-958c-695929768cae admin admin] RBAC: Authorizing `identity:delete_project(project_id=01c9a2e469214b2385f22d776e724021)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:49:26.183539 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79070698-149f-4262-958c-695929768cae admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:49:26.228881 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-79070698-149f-4262-958c-695929768cae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01c9a2e469214b2385f22d776e724021', 'request_id': 'req-79070698-149f-4262-958c-695929768cae'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:49:26.235081 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-79070698-149f-4262-958c-695929768cae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01c9a2e469214b2385f22d776e724021', 'request_id': 'req-79070698-149f-4262-958c-695929768cae'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:49:26.240666 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-79070698-149f-4262-958c-695929768cae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01c9a2e469214b2385f22d776e724021', 'request_id': 'req-79070698-149f-4262-958c-695929768cae'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:49:26.253610 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-79070698-149f-4262-958c-695929768cae admin admin] The token cache is being invalidate because project 01c9a2e469214b2385f22d776e724021 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:49:26.253902 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-79070698-149f-4262-958c-695929768cae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:49:26.254543 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-79070698-149f-4262-958c-695929768cae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:49:26.256092 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1407/2823] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Aug 6 01:49:26 2022] DELETE /identity/v3/projects/01c9a2e469214b2385f22d776e724021 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:49:26.261829 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa9a9c7-6f23-4642-865f-9e605964b49e None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:49:26.262292 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa9a9c7-6f23-4642-865f-9e605964b49e None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:49:26.262409 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa9a9c7-6f23-4642-865f-9e605964b49e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:49:26.281380 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-4fa9a9c7-6f23-4642-865f-9e605964b49e None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:49:26.282579 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-4fa9a9c7-6f23-4642-865f-9e605964b49e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:49:26.340574 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1417/2824] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:49:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:49:26.348337 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89afba2-0da9-460a-90db-ac778e72fad2 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:49:26.348571 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89afba2-0da9-460a-90db-ac778e72fad2 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:49:26.348857 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89afba2-0da9-460a-90db-ac778e72fad2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:49:26.368986 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-a89afba2-0da9-460a-90db-ac778e72fad2 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:49:26.370227 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-a89afba2-0da9-460a-90db-ac778e72fad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:49:26.378756 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1408/2825] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:49:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:49:26.383055 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa9a9c7-6f23-4642-865f-9e605964b49e None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:49:26.393366 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa9a9c7-6f23-4642-865f-9e605964b49e None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:49:26.398016 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0a5c14-ba5e-423e-af0d-7ee474acc7bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:49:26.398631 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0a5c14-ba5e-423e-af0d-7ee474acc7bb admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:49:26.398915 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0a5c14-ba5e-423e-af0d-7ee474acc7bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:49:26.399193 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0a5c14-ba5e-423e-af0d-7ee474acc7bb admin admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:49:26.399947 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0a5c14-ba5e-423e-af0d-7ee474acc7bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:49:26.400243 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0a5c14-ba5e-423e-af0d-7ee474acc7bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:49:26.401887 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0a5c14-ba5e-423e-af0d-7ee474acc7bb admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:49:26.413801 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1418/2826] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Aug 6 01:49:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:49:26.418436 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89afba2-0da9-460a-90db-ac778e72fad2 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:49:26.427800 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89afba2-0da9-460a-90db-ac778e72fad2 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:49:26.433516 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafa8b45-0e94-42df-9bbd-8787e7fe74c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:49:26.434124 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafa8b45-0e94-42df-9bbd-8787e7fe74c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:49:26.434408 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafa8b45-0e94-42df-9bbd-8787e7fe74c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:49:26.434821 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafa8b45-0e94-42df-9bbd-8787e7fe74c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:49:26.435503 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafa8b45-0e94-42df-9bbd-8787e7fe74c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:49:26.437440 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafa8b45-0e94-42df-9bbd-8787e7fe74c7 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:49:26.465169 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1409/2827] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Aug 6 01:49:26 2022] POST /identity/v3/projects => generated 377 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:49:26.469318 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0a5c14-ba5e-423e-af0d-7ee474acc7bb admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:49:26.479325 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0a5c14-ba5e-423e-af0d-7ee474acc7bb admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:49:26.535108 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8287f2-27fe-4df4-9ad5-a52696a7d3a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:49:26.535727 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8287f2-27fe-4df4-9ad5-a52696a7d3a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:49:26.536016 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8287f2-27fe-4df4-9ad5-a52696a7d3a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:49:26.536294 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8287f2-27fe-4df4-9ad5-a52696a7d3a8 admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:49:26.536962 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8287f2-27fe-4df4-9ad5-a52696a7d3a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:49:26.538904 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8287f2-27fe-4df4-9ad5-a52696a7d3a8 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:49:26.560449 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1419/2828] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:49:26 2022] POST /identity/v3/users => generated 289 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:49:26.564553 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafa8b45-0e94-42df-9bbd-8787e7fe74c7 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:49:26.574906 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafa8b45-0e94-42df-9bbd-8787e7fe74c7 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:49:26.579478 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ae7486-cecd-487f-8a50-706fae4d2a02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:49:26.580083 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ae7486-cecd-487f-8a50-706fae4d2a02 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:49:26.580363 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ae7486-cecd-487f-8a50-706fae4d2a02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:49:26.580636 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ae7486-cecd-487f-8a50-706fae4d2a02 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:49:26.581348 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ae7486-cecd-487f-8a50-706fae4d2a02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:49:26.582920 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ae7486-cecd-487f-8a50-706fae4d2a02 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:49:26.592711 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1410/2829] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:49:26 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 01:49:26.596490 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8287f2-27fe-4df4-9ad5-a52696a7d3a8 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:49:26.607025 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8287f2-27fe-4df4-9ad5-a52696a7d3a8 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:49:26.611549 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ae8a4e-0619-4129-b3cc-75c946709e31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:49:26.612158 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ae8a4e-0619-4129-b3cc-75c946709e31 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:49:26.612444 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ae8a4e-0619-4129-b3cc-75c946709e31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:49:26.612718 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ae8a4e-0619-4129-b3cc-75c946709e31 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:49:26.613441 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ae8a4e-0619-4129-b3cc-75c946709e31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:49:26.615052 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ae8a4e-0619-4129-b3cc-75c946709e31 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:49:26.627611 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1420/2830] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:49:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:49:26.631623 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ae7486-cecd-487f-8a50-706fae4d2a02 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:49:26.641640 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ae7486-cecd-487f-8a50-706fae4d2a02 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:49:26.646242 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f6bb8c-3d18-49c9-b528-38ebad5a6ecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:49:26.646894 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f6bb8c-3d18-49c9-b528-38ebad5a6ecf admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:49:26.647173 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f6bb8c-3d18-49c9-b528-38ebad5a6ecf admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:49:26.647449 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f6bb8c-3d18-49c9-b528-38ebad5a6ecf admin admin] PATH_INFO: `/v3/projects/e6f24b77fa7546d0a7aeb9b48fbcb954/users/fa7e4ea600bc425d9d38424237a9785f/roles/61b3b93d37a047e4bb8369b46fe20db9` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:49:26.661869 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f6bb8c-3d18-49c9-b528-38ebad5a6ecf admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6f24b77fa7546d0a7aeb9b48fbcb954, user_id=fa7e4ea600bc425d9d38424237a9785f, role_id=61b3b93d37a047e4bb8369b46fe20db9)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:49:26.665360 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f6bb8c-3d18-49c9-b528-38ebad5a6ecf admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:49:26.678005 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1411/2831] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:49:26 2022] PUT /identity/v3/projects/e6f24b77fa7546d0a7aeb9b48fbcb954/users/fa7e4ea600bc425d9d38424237a9785f/roles/61b3b93d37a047e4bb8369b46fe20db9 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:49:26.688568 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2daf21-ea00-474c-8bfd-d4c6dd273bd2 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:49:26.688863 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2daf21-ea00-474c-8bfd-d4c6dd273bd2 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:49:26.689175 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2daf21-ea00-474c-8bfd-d4c6dd273bd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:49:26.716076 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-fe2daf21-ea00-474c-8bfd-d4c6dd273bd2 None None] MFA Rules not processed for user `fa7e4ea600bc425d9d38424237a9785f`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:49:26.717318 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-fe2daf21-ea00-474c-8bfd-d4c6dd273bd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:49:26.759843 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:49:26.784503 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1421/2832] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:49:26 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:49:26.788854 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f6bb8c-3d18-49c9-b528-38ebad5a6ecf admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:49:26.833738 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:49:26.853274 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f6bb8c-3d18-49c9-b528-38ebad5a6ecf admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:49:26.858012 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665586fb-f304-48e8-a65f-8d13f35f44dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:49:26.858568 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665586fb-f304-48e8-a65f-8d13f35f44dc admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:49:26.858851 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665586fb-f304-48e8-a65f-8d13f35f44dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:49:26.859129 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665586fb-f304-48e8-a65f-8d13f35f44dc admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:49:26.859816 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665586fb-f304-48e8-a65f-8d13f35f44dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:49:26.861840 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665586fb-f304-48e8-a65f-8d13f35f44dc admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:49:26.888521 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1412/2833] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Aug 6 01:49:26 2022] POST /identity/v3/projects => generated 377 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:49:26.892556 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2daf21-ea00-474c-8bfd-d4c6dd273bd2 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:49:26.903895 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2daf21-ea00-474c-8bfd-d4c6dd273bd2 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:49:26.959736 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cf7177-c0d1-4a95-9d3f-7325cda2fb24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:49:26.960362 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cf7177-c0d1-4a95-9d3f-7325cda2fb24 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:49:26.960670 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cf7177-c0d1-4a95-9d3f-7325cda2fb24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:49:26.960946 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cf7177-c0d1-4a95-9d3f-7325cda2fb24 admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:49:26.962156 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cf7177-c0d1-4a95-9d3f-7325cda2fb24 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:49:26.964280 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cf7177-c0d1-4a95-9d3f-7325cda2fb24 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:49:26.985864 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1422/2834] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:49:26 2022] POST /identity/v3/users => generated 295 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:49:26.989804 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665586fb-f304-48e8-a65f-8d13f35f44dc admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:49:27.000469 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665586fb-f304-48e8-a65f-8d13f35f44dc admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:49:27.005152 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e254a589-028b-4d0d-bf2f-b48bd5f75a28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:49:27.005771 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e254a589-028b-4d0d-bf2f-b48bd5f75a28 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:49:27.006059 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e254a589-028b-4d0d-bf2f-b48bd5f75a28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:49:27.006344 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e254a589-028b-4d0d-bf2f-b48bd5f75a28 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:49:27.007048 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e254a589-028b-4d0d-bf2f-b48bd5f75a28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:49:27.008724 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e254a589-028b-4d0d-bf2f-b48bd5f75a28 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:49:27.019178 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1413/2835] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:49:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:49:27.023262 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cf7177-c0d1-4a95-9d3f-7325cda2fb24 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:49:27.033718 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cf7177-c0d1-4a95-9d3f-7325cda2fb24 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:49:27.038320 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09612f39-c818-4bcc-a3e9-cc06073b9401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:49:27.038968 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09612f39-c818-4bcc-a3e9-cc06073b9401 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:49:27.039248 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09612f39-c818-4bcc-a3e9-cc06073b9401 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:49:27.039518 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09612f39-c818-4bcc-a3e9-cc06073b9401 admin admin] PATH_INFO: `/v3/projects/33a0740613074d86beceb581b3f46c92/users/ac21d20e66484178ad60347b135f4d09/roles/4559b9a0837249f895a13c70096f8d38` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:49:27.054731 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09612f39-c818-4bcc-a3e9-cc06073b9401 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33a0740613074d86beceb581b3f46c92, user_id=ac21d20e66484178ad60347b135f4d09, role_id=4559b9a0837249f895a13c70096f8d38)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:49:27.057785 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09612f39-c818-4bcc-a3e9-cc06073b9401 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:49:27.070247 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1423/2836] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:49:27 2022] PUT /identity/v3/projects/33a0740613074d86beceb581b3f46c92/users/ac21d20e66484178ad60347b135f4d09/roles/4559b9a0837249f895a13c70096f8d38 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:49:27.074283 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e254a589-028b-4d0d-bf2f-b48bd5f75a28 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:49:27.084371 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e254a589-028b-4d0d-bf2f-b48bd5f75a28 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:49:27.141548 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b21360a-0869-43ef-bed6-056dc29377a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:49:27.142457 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b21360a-0869-43ef-bed6-056dc29377a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:49:27.142937 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b21360a-0869-43ef-bed6-056dc29377a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:49:27.143221 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b21360a-0869-43ef-bed6-056dc29377a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:49:27.143918 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b21360a-0869-43ef-bed6-056dc29377a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:49:27.146187 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b21360a-0869-43ef-bed6-056dc29377a8 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:49:27.157505 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1414/2837] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:49:27 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:49:27.161687 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09612f39-c818-4bcc-a3e9-cc06073b9401 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:49:27.172530 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09612f39-c818-4bcc-a3e9-cc06073b9401 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:49:27.177142 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7403038-1959-463c-aee2-a36e3e8e6585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:49:27.177801 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7403038-1959-463c-aee2-a36e3e8e6585 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:49:27.178087 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7403038-1959-463c-aee2-a36e3e8e6585 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:49:27.178362 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7403038-1959-463c-aee2-a36e3e8e6585 admin admin] PATH_INFO: `/v3/projects/33a0740613074d86beceb581b3f46c92/users/ac21d20e66484178ad60347b135f4d09/roles/4559b9a0837249f895a13c70096f8d38` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:49:27.180304 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7403038-1959-463c-aee2-a36e3e8e6585 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33a0740613074d86beceb581b3f46c92, user_id=ac21d20e66484178ad60347b135f4d09, role_id=4559b9a0837249f895a13c70096f8d38)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:49:27.183150 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7403038-1959-463c-aee2-a36e3e8e6585 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:49:27.193243 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1424/2838] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:49:27 2022] PUT /identity/v3/projects/33a0740613074d86beceb581b3f46c92/users/ac21d20e66484178ad60347b135f4d09/roles/4559b9a0837249f895a13c70096f8d38 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:49:27.204744 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598cbf2a-5e15-43b3-b14e-89c40a3f64b3 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:49:27.205253 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598cbf2a-5e15-43b3-b14e-89c40a3f64b3 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:49:27.205367 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598cbf2a-5e15-43b3-b14e-89c40a3f64b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:49:27.235384 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-598cbf2a-5e15-43b3-b14e-89c40a3f64b3 None None] MFA Rules not processed for user `ac21d20e66484178ad60347b135f4d09`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:49:27.236586 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-598cbf2a-5e15-43b3-b14e-89c40a3f64b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:49:27.306112 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1415/2839] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:49:27 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:49:27.316825 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7403038-1959-463c-aee2-a36e3e8e6585 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:49:27.318766 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-c7403038-1959-463c-aee2-a36e3e8e6585 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:49:27.376117 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7403038-1959-463c-aee2-a36e3e8e6585 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:49:27.379752 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e82700a-c16e-4f84-8de6-55a45803e06e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1be1e6173e374a60b9e1fbf51a1ab571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:49:27.380365 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e82700a-c16e-4f84-8de6-55a45803e06e service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:49:27.380648 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e82700a-c16e-4f84-8de6-55a45803e06e service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:49:27.380922 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e82700a-c16e-4f84-8de6-55a45803e06e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:49:27.427803 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:49:27.438503 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e82700a-c16e-4f84-8de6-55a45803e06e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:49:27.439935 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e82700a-c16e-4f84-8de6-55a45803e06e service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:49:27.449864 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1425/2840] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:49:27 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:49:33.085754 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598cbf2a-5e15-43b3-b14e-89c40a3f64b3 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:49:33.087557 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-598cbf2a-5e15-43b3-b14e-89c40a3f64b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:49:33.099222 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598cbf2a-5e15-43b3-b14e-89c40a3f64b3 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:49:33.102997 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7b348f-66a1-408e-bd60-61fc53d42765 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1be1e6173e374a60b9e1fbf51a1ab571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:49:33.103602 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7b348f-66a1-408e-bd60-61fc53d42765 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:49:33.103884 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7b348f-66a1-408e-bd60-61fc53d42765 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:49:33.104161 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7b348f-66a1-408e-bd60-61fc53d42765 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:49:33.112789 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7b348f-66a1-408e-bd60-61fc53d42765 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:49:33.114246 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7b348f-66a1-408e-bd60-61fc53d42765 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:49:33.124518 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1416/2841] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:49:33 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:49:37.445199 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=5 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:49:43.120056 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=88,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:50:02.041890 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e82700a-c16e-4f84-8de6-55a45803e06e service cinder] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:50:02.103781 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e82700a-c16e-4f84-8de6-55a45803e06e service cinder] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:50:02.108440 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6f70ae-4441-46b5-b77d-ad5fce9a0142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:50:02.109094 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6f70ae-4441-46b5-b77d-ad5fce9a0142 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:50:02.109379 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6f70ae-4441-46b5-b77d-ad5fce9a0142 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:50:02.109655 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6f70ae-4441-46b5-b77d-ad5fce9a0142 admin admin] PATH_INFO: `/v3/users/fa7e4ea600bc425d9d38424237a9785f` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:50:02.110838 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6f70ae-4441-46b5-b77d-ad5fce9a0142 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa7e4ea600bc425d9d38424237a9785f)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:50:02.112783 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6f70ae-4441-46b5-b77d-ad5fce9a0142 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:50:02.158342 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-cb6f70ae-4441-46b5-b77d-ad5fce9a0142 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa7e4ea600bc425d9d38424237a9785f'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:50:02.164996 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-cb6f70ae-4441-46b5-b77d-ad5fce9a0142 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa7e4ea600bc425d9d38424237a9785f'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:50:02.165683 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-cb6f70ae-4441-46b5-b77d-ad5fce9a0142 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa7e4ea600bc425d9d38424237a9785f'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:50:02.166292 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-cb6f70ae-4441-46b5-b77d-ad5fce9a0142 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa7e4ea600bc425d9d38424237a9785f'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:50:02.177946 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-cb6f70ae-4441-46b5-b77d-ad5fce9a0142 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa7e4ea600bc425d9d38424237a9785f'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:50:02.190794 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1426/2842] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Aug 6 01:50:02 2022] DELETE /identity/v3/users/fa7e4ea600bc425d9d38424237a9785f => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:50:02.195985 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eb1cb7-30dc-4c70-8e80-e5b244ca5c7f None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:50:02.196510 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eb1cb7-30dc-4c70-8e80-e5b244ca5c7f None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:50:02.196734 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eb1cb7-30dc-4c70-8e80-e5b244ca5c7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:50:02.215208 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-72eb1cb7-30dc-4c70-8e80-e5b244ca5c7f None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:50:02.216413 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-72eb1cb7-30dc-4c70-8e80-e5b244ca5c7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:50:02.274443 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1417/2843] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:50:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:50:02.283912 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6f70ae-4441-46b5-b77d-ad5fce9a0142 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:50:02.285457 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-cb6f70ae-4441-46b5-b77d-ad5fce9a0142 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:50:02.341585 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6f70ae-4441-46b5-b77d-ad5fce9a0142 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:50:02.345378 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0f4df1-f94c-440e-9aca-d2ce504c4f74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:50:02.345998 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0f4df1-f94c-440e-9aca-d2ce504c4f74 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:50:02.346281 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0f4df1-f94c-440e-9aca-d2ce504c4f74 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:50:02.346559 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0f4df1-f94c-440e-9aca-d2ce504c4f74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:50:02.355659 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0f4df1-f94c-440e-9aca-d2ce504c4f74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:50:02.357109 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0f4df1-f94c-440e-9aca-d2ce504c4f74 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:50:02.366398 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1427/2844] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:50:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:50:02.639164 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eb1cb7-30dc-4c70-8e80-e5b244ca5c7f None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:50:02.640610 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-72eb1cb7-30dc-4c70-8e80-e5b244ca5c7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:50:02.653188 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eb1cb7-30dc-4c70-8e80-e5b244ca5c7f None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:50:02.658946 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67b2a39-6a61-42f3-9735-796989e1ed47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:50:02.659592 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67b2a39-6a61-42f3-9735-796989e1ed47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:50:02.659875 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67b2a39-6a61-42f3-9735-796989e1ed47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:50:02.660172 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67b2a39-6a61-42f3-9735-796989e1ed47 admin admin] PATH_INFO: `/v3/projects/e6f24b77fa7546d0a7aeb9b48fbcb954` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:50:02.661353 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67b2a39-6a61-42f3-9735-796989e1ed47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6f24b77fa7546d0a7aeb9b48fbcb954)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:50:02.664645 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67b2a39-6a61-42f3-9735-796989e1ed47 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:50:02.712427 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-f67b2a39-6a61-42f3-9735-796989e1ed47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6f24b77fa7546d0a7aeb9b48fbcb954', 'request_id': 'req-f67b2a39-6a61-42f3-9735-796989e1ed47'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:50:02.718506 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-f67b2a39-6a61-42f3-9735-796989e1ed47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6f24b77fa7546d0a7aeb9b48fbcb954', 'request_id': 'req-f67b2a39-6a61-42f3-9735-796989e1ed47'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:50:02.724315 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-f67b2a39-6a61-42f3-9735-796989e1ed47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6f24b77fa7546d0a7aeb9b48fbcb954', 'request_id': 'req-f67b2a39-6a61-42f3-9735-796989e1ed47'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:50:02.737397 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-f67b2a39-6a61-42f3-9735-796989e1ed47 admin admin] The token cache is being invalidate because project e6f24b77fa7546d0a7aeb9b48fbcb954 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:50:02.737693 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-f67b2a39-6a61-42f3-9735-796989e1ed47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:50:02.738317 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-f67b2a39-6a61-42f3-9735-796989e1ed47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:50:02.739847 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1418/2845] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Aug 6 01:50:02 2022] DELETE /identity/v3/projects/e6f24b77fa7546d0a7aeb9b48fbcb954 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:50:02.744474 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0f4df1-f94c-440e-9aca-d2ce504c4f74 service neutron] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:50:02.754255 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0f4df1-f94c-440e-9aca-d2ce504c4f74 service neutron] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:50:02.772120 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:50:02.811946 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca640348-ab07-4491-9aa4-19fbbcd35a6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:50:02.812507 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca640348-ab07-4491-9aa4-19fbbcd35a6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:50:02.812791 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca640348-ab07-4491-9aa4-19fbbcd35a6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:50:02.813108 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca640348-ab07-4491-9aa4-19fbbcd35a6f admin admin] PATH_INFO: `/v3/users/ac21d20e66484178ad60347b135f4d09` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:50:02.814274 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca640348-ab07-4491-9aa4-19fbbcd35a6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac21d20e66484178ad60347b135f4d09)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:50:02.816196 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca640348-ab07-4491-9aa4-19fbbcd35a6f admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:50:02.861621 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-ca640348-ab07-4491-9aa4-19fbbcd35a6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac21d20e66484178ad60347b135f4d09'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:50:02.868864 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-ca640348-ab07-4491-9aa4-19fbbcd35a6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac21d20e66484178ad60347b135f4d09'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:50:02.869549 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-ca640348-ab07-4491-9aa4-19fbbcd35a6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac21d20e66484178ad60347b135f4d09'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:50:02.870163 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-ca640348-ab07-4491-9aa4-19fbbcd35a6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac21d20e66484178ad60347b135f4d09'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:50:02.881970 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-ca640348-ab07-4491-9aa4-19fbbcd35a6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac21d20e66484178ad60347b135f4d09'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:50:02.894193 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1428/2846] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Aug 6 01:50:02 2022] DELETE /identity/v3/users/ac21d20e66484178ad60347b135f4d09 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:50:03.166115 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67b2a39-6a61-42f3-9735-796989e1ed47 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:50:03.167578 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-f67b2a39-6a61-42f3-9735-796989e1ed47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:50:03.181006 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67b2a39-6a61-42f3-9735-796989e1ed47 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:50:03.244481 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558208ba-32cb-47b6-bcf0-7268aaef1c09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:50:03.245184 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558208ba-32cb-47b6-bcf0-7268aaef1c09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:50:03.245418 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558208ba-32cb-47b6-bcf0-7268aaef1c09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:50:03.245695 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558208ba-32cb-47b6-bcf0-7268aaef1c09 admin admin] PATH_INFO: `/v3/projects/33a0740613074d86beceb581b3f46c92` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:50:03.246849 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558208ba-32cb-47b6-bcf0-7268aaef1c09 admin admin] RBAC: Authorizing `identity:delete_project(project_id=33a0740613074d86beceb581b3f46c92)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:50:03.248991 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558208ba-32cb-47b6-bcf0-7268aaef1c09 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:50:03.280031 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:50:03.299607 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-558208ba-32cb-47b6-bcf0-7268aaef1c09 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33a0740613074d86beceb581b3f46c92', 'request_id': 'req-558208ba-32cb-47b6-bcf0-7268aaef1c09'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:50:03.306293 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-558208ba-32cb-47b6-bcf0-7268aaef1c09 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33a0740613074d86beceb581b3f46c92', 'request_id': 'req-558208ba-32cb-47b6-bcf0-7268aaef1c09'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:50:03.311705 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-558208ba-32cb-47b6-bcf0-7268aaef1c09 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33a0740613074d86beceb581b3f46c92', 'request_id': 'req-558208ba-32cb-47b6-bcf0-7268aaef1c09'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:50:03.324108 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-558208ba-32cb-47b6-bcf0-7268aaef1c09 admin admin] The token cache is being invalidate because project 33a0740613074d86beceb581b3f46c92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:50:03.324401 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-558208ba-32cb-47b6-bcf0-7268aaef1c09 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:50:03.325067 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-558208ba-32cb-47b6-bcf0-7268aaef1c09 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:50:03.326605 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1419/2847] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Aug 6 01:50:03 2022] DELETE /identity/v3/projects/33a0740613074d86beceb581b3f46c92 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:50:03.331457 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a209e0a-aa0f-46fd-9c1e-70f6adc5e608 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:50:03.331919 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a209e0a-aa0f-46fd-9c1e-70f6adc5e608 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:50:03.332033 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a209e0a-aa0f-46fd-9c1e-70f6adc5e608 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:50:03.351107 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-0a209e0a-aa0f-46fd-9c1e-70f6adc5e608 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:50:03.352960 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-0a209e0a-aa0f-46fd-9c1e-70f6adc5e608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:50:03.420231 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1429/2848] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:50:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:50:03.422240 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b942b3a-f3a5-4b41-8b4a-917d62a0f36e None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:50:03.422505 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b942b3a-f3a5-4b41-8b4a-917d62a0f36e None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:50:03.422888 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b942b3a-f3a5-4b41-8b4a-917d62a0f36e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:50:03.442816 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-8b942b3a-f3a5-4b41-8b4a-917d62a0f36e None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:50:03.444026 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-8b942b3a-f3a5-4b41-8b4a-917d62a0f36e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:50:03.452885 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1420/2849] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:50:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:50:03.457103 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a209e0a-aa0f-46fd-9c1e-70f6adc5e608 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:50:03.468088 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a209e0a-aa0f-46fd-9c1e-70f6adc5e608 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:50:03.472853 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d4229d-47b3-489c-af2f-8eaadd79189b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:50:03.473530 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d4229d-47b3-489c-af2f-8eaadd79189b admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:50:03.473839 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d4229d-47b3-489c-af2f-8eaadd79189b admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:50:03.474094 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d4229d-47b3-489c-af2f-8eaadd79189b admin admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:50:03.474792 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d4229d-47b3-489c-af2f-8eaadd79189b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:50:03.475125 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d4229d-47b3-489c-af2f-8eaadd79189b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:50:03.476878 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d4229d-47b3-489c-af2f-8eaadd79189b admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:50:03.488724 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1430/2850] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Aug 6 01:50:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:50:03.493067 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b942b3a-f3a5-4b41-8b4a-917d62a0f36e None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:50:03.502686 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b942b3a-f3a5-4b41-8b4a-917d62a0f36e None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:50:03.508072 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb63add-0651-4820-803f-493d416b19e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:50:03.508678 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb63add-0651-4820-803f-493d416b19e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:50:03.508966 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb63add-0651-4820-803f-493d416b19e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:50:03.509385 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb63add-0651-4820-803f-493d416b19e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:50:03.510056 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb63add-0651-4820-803f-493d416b19e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:50:03.511958 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb63add-0651-4820-803f-493d416b19e3 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:50:03.539679 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1421/2851] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Aug 6 01:50:03 2022] POST /identity/v3/projects => generated 387 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:50:03.543931 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d4229d-47b3-489c-af2f-8eaadd79189b admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:50:03.553898 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d4229d-47b3-489c-af2f-8eaadd79189b admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:50:03.579998 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:50:03.611408 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2b2956-7d84-4e74-ae6c-55601b292a9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:50:03.611949 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2b2956-7d84-4e74-ae6c-55601b292a9c admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:50:03.612225 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2b2956-7d84-4e74-ae6c-55601b292a9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:50:03.612499 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2b2956-7d84-4e74-ae6c-55601b292a9c admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:50:03.613187 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2b2956-7d84-4e74-ae6c-55601b292a9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:50:03.615084 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2b2956-7d84-4e74-ae6c-55601b292a9c admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:50:03.636023 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1431/2852] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:50:03 2022] POST /identity/v3/users => generated 294 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:50:03.640231 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb63add-0651-4820-803f-493d416b19e3 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:50:03.650991 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb63add-0651-4820-803f-493d416b19e3 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:50:03.655636 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f85910-abe9-4dae-8513-f68f7f90400a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:50:03.656223 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f85910-abe9-4dae-8513-f68f7f90400a admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:50:03.656502 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f85910-abe9-4dae-8513-f68f7f90400a admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:50:03.656772 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f85910-abe9-4dae-8513-f68f7f90400a admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:50:03.657493 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f85910-abe9-4dae-8513-f68f7f90400a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:50:03.659071 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f85910-abe9-4dae-8513-f68f7f90400a admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:50:03.669067 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1422/2853] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:50:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:50:03.672869 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2b2956-7d84-4e74-ae6c-55601b292a9c admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:50:03.683382 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2b2956-7d84-4e74-ae6c-55601b292a9c admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:50:03.687988 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d977295d-39dc-4eb6-90a2-202023355496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:50:03.688607 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d977295d-39dc-4eb6-90a2-202023355496 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:50:03.688889 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d977295d-39dc-4eb6-90a2-202023355496 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:50:03.689200 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d977295d-39dc-4eb6-90a2-202023355496 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:50:03.689888 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d977295d-39dc-4eb6-90a2-202023355496 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:50:03.691461 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d977295d-39dc-4eb6-90a2-202023355496 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:50:03.702627 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1432/2854] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:50:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:50:03.706716 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f85910-abe9-4dae-8513-f68f7f90400a admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:50:03.716436 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f85910-abe9-4dae-8513-f68f7f90400a admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:50:03.721061 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de01e38c-7483-47cd-966e-a5bbe02fdf38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:50:03.721717 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de01e38c-7483-47cd-966e-a5bbe02fdf38 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:50:03.721995 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de01e38c-7483-47cd-966e-a5bbe02fdf38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:50:03.722266 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de01e38c-7483-47cd-966e-a5bbe02fdf38 admin admin] PATH_INFO: `/v3/projects/1509f97a50c943a6a08c73e517533ead/users/c15fc201f19a4bb283d3469b44d8f0e2/roles/61b3b93d37a047e4bb8369b46fe20db9` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:50:03.736266 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de01e38c-7483-47cd-966e-a5bbe02fdf38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1509f97a50c943a6a08c73e517533ead, user_id=c15fc201f19a4bb283d3469b44d8f0e2, role_id=61b3b93d37a047e4bb8369b46fe20db9)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:50:03.738908 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de01e38c-7483-47cd-966e-a5bbe02fdf38 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:50:03.751939 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1423/2855] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:50:03 2022] PUT /identity/v3/projects/1509f97a50c943a6a08c73e517533ead/users/c15fc201f19a4bb283d3469b44d8f0e2/roles/61b3b93d37a047e4bb8369b46fe20db9 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:50:03.764027 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9447aac5-5375-461f-aa03-d1f6eb5c0a67 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:50:03.764312 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9447aac5-5375-461f-aa03-d1f6eb5c0a67 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:50:03.764586 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9447aac5-5375-461f-aa03-d1f6eb5c0a67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:50:03.792035 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-9447aac5-5375-461f-aa03-d1f6eb5c0a67 None None] MFA Rules not processed for user `c15fc201f19a4bb283d3469b44d8f0e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:50:03.793259 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-9447aac5-5375-461f-aa03-d1f6eb5c0a67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:50:03.856364 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1433/2856] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:50:03 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:50:03.860936 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de01e38c-7483-47cd-966e-a5bbe02fdf38 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:50:03.922019 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de01e38c-7483-47cd-966e-a5bbe02fdf38 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:50:03.926569 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d533f9-5cdf-4077-9f73-4227b3e6d3f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:50:03.927174 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d533f9-5cdf-4077-9f73-4227b3e6d3f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:50:03.927456 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d533f9-5cdf-4077-9f73-4227b3e6d3f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:50:03.927731 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d533f9-5cdf-4077-9f73-4227b3e6d3f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:50:03.928400 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d533f9-5cdf-4077-9f73-4227b3e6d3f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:50:03.930851 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d533f9-5cdf-4077-9f73-4227b3e6d3f1 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:50:03.956992 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1424/2857] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Aug 6 01:50:03 2022] POST /identity/v3/projects => generated 387 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:50:03.961071 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9447aac5-5375-461f-aa03-d1f6eb5c0a67 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:50:03.972258 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9447aac5-5375-461f-aa03-d1f6eb5c0a67 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:50:04.028260 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393aaa98-5795-42c6-9dea-e5473ec43922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:50:04.028881 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393aaa98-5795-42c6-9dea-e5473ec43922 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:50:04.029198 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393aaa98-5795-42c6-9dea-e5473ec43922 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:50:04.029481 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393aaa98-5795-42c6-9dea-e5473ec43922 admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:50:04.030158 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393aaa98-5795-42c6-9dea-e5473ec43922 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:50:04.032075 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393aaa98-5795-42c6-9dea-e5473ec43922 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:50:04.053629 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1434/2858] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:50:03 2022] POST /identity/v3/users => generated 300 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:50:04.058290 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d533f9-5cdf-4077-9f73-4227b3e6d3f1 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:50:04.068778 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d533f9-5cdf-4077-9f73-4227b3e6d3f1 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:50:04.074554 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f04ce9-dfa3-42ef-b142-ba5505ffc170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:50:04.075180 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f04ce9-dfa3-42ef-b142-ba5505ffc170 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:50:04.075458 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f04ce9-dfa3-42ef-b142-ba5505ffc170 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:50:04.075735 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f04ce9-dfa3-42ef-b142-ba5505ffc170 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:50:04.076427 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f04ce9-dfa3-42ef-b142-ba5505ffc170 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:50:04.078057 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f04ce9-dfa3-42ef-b142-ba5505ffc170 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:50:04.089594 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1425/2859] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:50:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:50:04.094220 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393aaa98-5795-42c6-9dea-e5473ec43922 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:50:04.104419 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393aaa98-5795-42c6-9dea-e5473ec43922 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:50:04.109028 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904ebacd-7bdf-4543-ab9d-cf5983e869df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:50:04.109708 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904ebacd-7bdf-4543-ab9d-cf5983e869df admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:50:04.109991 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904ebacd-7bdf-4543-ab9d-cf5983e869df admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:50:04.110270 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904ebacd-7bdf-4543-ab9d-cf5983e869df admin admin] PATH_INFO: `/v3/projects/e11f68a514564dc4b03c5c1c21c13e4f/users/37d069313e4a4d169321ec12d078c8ef/roles/4559b9a0837249f895a13c70096f8d38` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:50:04.124957 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904ebacd-7bdf-4543-ab9d-cf5983e869df admin admin] RBAC: Authorizing `identity:create_grant(project_id=e11f68a514564dc4b03c5c1c21c13e4f, user_id=37d069313e4a4d169321ec12d078c8ef, role_id=4559b9a0837249f895a13c70096f8d38)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:50:04.127638 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904ebacd-7bdf-4543-ab9d-cf5983e869df admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:50:04.141111 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1435/2860] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:50:04 2022] PUT /identity/v3/projects/e11f68a514564dc4b03c5c1c21c13e4f/users/37d069313e4a4d169321ec12d078c8ef/roles/4559b9a0837249f895a13c70096f8d38 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:50:04.145183 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f04ce9-dfa3-42ef-b142-ba5505ffc170 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:50:04.155780 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f04ce9-dfa3-42ef-b142-ba5505ffc170 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:50:04.175380 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:50:04.213801 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba2ef8f-ad5e-4c1e-b187-76d93db538bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:50:04.214414 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba2ef8f-ad5e-4c1e-b187-76d93db538bb admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:50:04.214698 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba2ef8f-ad5e-4c1e-b187-76d93db538bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:50:04.214974 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba2ef8f-ad5e-4c1e-b187-76d93db538bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:50:04.215657 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba2ef8f-ad5e-4c1e-b187-76d93db538bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:50:04.217262 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba2ef8f-ad5e-4c1e-b187-76d93db538bb admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:50:04.228916 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1426/2861] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:50:04 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:50:04.232929 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904ebacd-7bdf-4543-ab9d-cf5983e869df admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:50:04.238386 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:50:04.245299 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904ebacd-7bdf-4543-ab9d-cf5983e869df admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:50:04.250431 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a602bd7b-22a3-4b68-8d7f-3d2b8a402d0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:50:04.251090 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a602bd7b-22a3-4b68-8d7f-3d2b8a402d0b admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:50:04.251367 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a602bd7b-22a3-4b68-8d7f-3d2b8a402d0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:50:04.251639 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a602bd7b-22a3-4b68-8d7f-3d2b8a402d0b admin admin] PATH_INFO: `/v3/projects/e11f68a514564dc4b03c5c1c21c13e4f/users/37d069313e4a4d169321ec12d078c8ef/roles/4559b9a0837249f895a13c70096f8d38` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:50:04.253644 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a602bd7b-22a3-4b68-8d7f-3d2b8a402d0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e11f68a514564dc4b03c5c1c21c13e4f, user_id=37d069313e4a4d169321ec12d078c8ef, role_id=4559b9a0837249f895a13c70096f8d38)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:50:04.256283 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a602bd7b-22a3-4b68-8d7f-3d2b8a402d0b admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:50:04.266910 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1436/2862] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:50:04 2022] PUT /identity/v3/projects/e11f68a514564dc4b03c5c1c21c13e4f/users/37d069313e4a4d169321ec12d078c8ef/roles/4559b9a0837249f895a13c70096f8d38 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:50:04.277627 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f415e2e0-d245-4f63-8be5-196548d95b8e None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:50:04.278222 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f415e2e0-d245-4f63-8be5-196548d95b8e None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:50:04.278222 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f415e2e0-d245-4f63-8be5-196548d95b8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:50:04.306772 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-f415e2e0-d245-4f63-8be5-196548d95b8e None None] MFA Rules not processed for user `37d069313e4a4d169321ec12d078c8ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:50:04.307983 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-f415e2e0-d245-4f63-8be5-196548d95b8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:50:04.389357 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1427/2863] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:50:04 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:50:04.405670 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a602bd7b-22a3-4b68-8d7f-3d2b8a402d0b admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:50:04.407606 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-a602bd7b-22a3-4b68-8d7f-3d2b8a402d0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:50:04.466171 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a602bd7b-22a3-4b68-8d7f-3d2b8a402d0b admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:50:04.469897 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2a2ec6-5fb9-42be-a3da-8c78fe828e98 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1be1e6173e374a60b9e1fbf51a1ab571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:50:04.470597 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2a2ec6-5fb9-42be-a3da-8c78fe828e98 service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:50:04.470885 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2a2ec6-5fb9-42be-a3da-8c78fe828e98 service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:50:04.471162 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2a2ec6-5fb9-42be-a3da-8c78fe828e98 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:50:04.480444 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2a2ec6-5fb9-42be-a3da-8c78fe828e98 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:50:04.481901 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2a2ec6-5fb9-42be-a3da-8c78fe828e98 service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:50:04.491639 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1437/2864] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:50:04 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:50:04.521679 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f415e2e0-d245-4f63-8be5-196548d95b8e None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:50:04.584719 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f415e2e0-d245-4f63-8be5-196548d95b8e None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:50:04.589914 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa2a3e2-c15d-4c88-8ac7-89065d8574a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:50:04.590597 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa2a3e2-c15d-4c88-8ac7-89065d8574a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:50:04.590885 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa2a3e2-c15d-4c88-8ac7-89065d8574a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:50:04.591167 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa2a3e2-c15d-4c88-8ac7-89065d8574a1 admin admin] PATH_INFO: `/v3/users/c15fc201f19a4bb283d3469b44d8f0e2` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:50:04.592355 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa2a3e2-c15d-4c88-8ac7-89065d8574a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c15fc201f19a4bb283d3469b44d8f0e2)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:50:04.594332 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa2a3e2-c15d-4c88-8ac7-89065d8574a1 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:50:04.640932 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-cfa2a3e2-c15d-4c88-8ac7-89065d8574a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c15fc201f19a4bb283d3469b44d8f0e2'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:50:04.641600 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-cfa2a3e2-c15d-4c88-8ac7-89065d8574a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c15fc201f19a4bb283d3469b44d8f0e2'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:50:04.642196 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-cfa2a3e2-c15d-4c88-8ac7-89065d8574a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c15fc201f19a4bb283d3469b44d8f0e2'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:50:04.643313 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:50:04.656441 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-cfa2a3e2-c15d-4c88-8ac7-89065d8574a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c15fc201f19a4bb283d3469b44d8f0e2'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:50:04.663210 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-cfa2a3e2-c15d-4c88-8ac7-89065d8574a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c15fc201f19a4bb283d3469b44d8f0e2'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:50:04.676312 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1428/2865] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Aug 6 01:50:04 2022] DELETE /identity/v3/users/c15fc201f19a4bb283d3469b44d8f0e2 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:50:04.681107 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20717ed7-4631-4766-8554-27b782137f64 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:50:04.681402 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20717ed7-4631-4766-8554-27b782137f64 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:50:04.681680 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20717ed7-4631-4766-8554-27b782137f64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:50:04.700665 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-20717ed7-4631-4766-8554-27b782137f64 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:50:04.701910 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-20717ed7-4631-4766-8554-27b782137f64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:50:04.760907 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1438/2866] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:50:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:50:04.770517 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa2a3e2-c15d-4c88-8ac7-89065d8574a1 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:50:04.771919 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-cfa2a3e2-c15d-4c88-8ac7-89065d8574a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:50:04.828981 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa2a3e2-c15d-4c88-8ac7-89065d8574a1 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:50:04.832885 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bbdfd7-3e2a-406a-85a2-e9ecb4fc4292 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:50:04.833556 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bbdfd7-3e2a-406a-85a2-e9ecb4fc4292 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:50:04.833841 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bbdfd7-3e2a-406a-85a2-e9ecb4fc4292 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:50:04.834120 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bbdfd7-3e2a-406a-85a2-e9ecb4fc4292 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:50:04.843985 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bbdfd7-3e2a-406a-85a2-e9ecb4fc4292 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:50:04.845431 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bbdfd7-3e2a-406a-85a2-e9ecb4fc4292 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:50:04.854889 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1429/2867] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:50:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:50:05.118979 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20717ed7-4631-4766-8554-27b782137f64 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:50:05.120473 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-20717ed7-4631-4766-8554-27b782137f64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:50:05.133704 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20717ed7-4631-4766-8554-27b782137f64 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:50:05.138592 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73004328-48c6-4259-ad8c-78483463d418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:50:05.139262 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73004328-48c6-4259-ad8c-78483463d418 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:50:05.139540 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73004328-48c6-4259-ad8c-78483463d418 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:50:05.139812 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73004328-48c6-4259-ad8c-78483463d418 admin admin] PATH_INFO: `/v3/projects/1509f97a50c943a6a08c73e517533ead` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:50:05.141008 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73004328-48c6-4259-ad8c-78483463d418 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1509f97a50c943a6a08c73e517533ead)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:50:05.142949 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73004328-48c6-4259-ad8c-78483463d418 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:50:05.190085 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-73004328-48c6-4259-ad8c-78483463d418 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1509f97a50c943a6a08c73e517533ead', 'request_id': 'req-73004328-48c6-4259-ad8c-78483463d418'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:50:05.196958 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-73004328-48c6-4259-ad8c-78483463d418 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1509f97a50c943a6a08c73e517533ead', 'request_id': 'req-73004328-48c6-4259-ad8c-78483463d418'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:50:05.202470 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-73004328-48c6-4259-ad8c-78483463d418 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1509f97a50c943a6a08c73e517533ead', 'request_id': 'req-73004328-48c6-4259-ad8c-78483463d418'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:50:05.214568 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-73004328-48c6-4259-ad8c-78483463d418 admin admin] The token cache is being invalidate because project 1509f97a50c943a6a08c73e517533ead was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:50:05.214854 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-73004328-48c6-4259-ad8c-78483463d418 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:50:05.215506 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-73004328-48c6-4259-ad8c-78483463d418 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:50:05.217016 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1439/2868] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Aug 6 01:50:05 2022] DELETE /identity/v3/projects/1509f97a50c943a6a08c73e517533ead => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:50:05.220986 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bbdfd7-3e2a-406a-85a2-e9ecb4fc4292 service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:50:05.231778 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bbdfd7-3e2a-406a-85a2-e9ecb4fc4292 service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:50:05.288272 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9182ce94-0c4c-4bcf-bb8f-73de10aa4727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:50:05.289687 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9182ce94-0c4c-4bcf-bb8f-73de10aa4727 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:50:05.290083 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9182ce94-0c4c-4bcf-bb8f-73de10aa4727 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:50:05.290371 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9182ce94-0c4c-4bcf-bb8f-73de10aa4727 admin admin] PATH_INFO: `/v3/users/37d069313e4a4d169321ec12d078c8ef` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:50:05.291562 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9182ce94-0c4c-4bcf-bb8f-73de10aa4727 admin admin] RBAC: Authorizing `identity:delete_user(user_id=37d069313e4a4d169321ec12d078c8ef)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:50:05.293627 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9182ce94-0c4c-4bcf-bb8f-73de10aa4727 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:50:05.338478 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-9182ce94-0c4c-4bcf-bb8f-73de10aa4727 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37d069313e4a4d169321ec12d078c8ef'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:50:05.339111 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-9182ce94-0c4c-4bcf-bb8f-73de10aa4727 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37d069313e4a4d169321ec12d078c8ef'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:50:05.339703 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-9182ce94-0c4c-4bcf-bb8f-73de10aa4727 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37d069313e4a4d169321ec12d078c8ef'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:50:05.351127 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-9182ce94-0c4c-4bcf-bb8f-73de10aa4727 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37d069313e4a4d169321ec12d078c8ef'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:50:05.356067 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:50:05.360005 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-9182ce94-0c4c-4bcf-bb8f-73de10aa4727 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37d069313e4a4d169321ec12d078c8ef'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:50:05.373158 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1430/2869] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Aug 6 01:50:05 2022] DELETE /identity/v3/users/37d069313e4a4d169321ec12d078c8ef => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:50:05.874458 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73004328-48c6-4259-ad8c-78483463d418 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:50:05.875947 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-73004328-48c6-4259-ad8c-78483463d418 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:50:05.890115 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73004328-48c6-4259-ad8c-78483463d418 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:50:05.903410 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:50:05.950739 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b568568-55de-4e4c-83c8-6ba1e15cbc39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:50:05.951559 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b568568-55de-4e4c-83c8-6ba1e15cbc39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:50:05.951842 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b568568-55de-4e4c-83c8-6ba1e15cbc39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:50:05.952116 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b568568-55de-4e4c-83c8-6ba1e15cbc39 admin admin] PATH_INFO: `/v3/projects/e11f68a514564dc4b03c5c1c21c13e4f` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:50:05.953293 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b568568-55de-4e4c-83c8-6ba1e15cbc39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e11f68a514564dc4b03c5c1c21c13e4f)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:50:05.955189 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b568568-55de-4e4c-83c8-6ba1e15cbc39 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:50:06.002338 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-2b568568-55de-4e4c-83c8-6ba1e15cbc39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e11f68a514564dc4b03c5c1c21c13e4f', 'request_id': 'req-2b568568-55de-4e4c-83c8-6ba1e15cbc39'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:50:06.009219 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-2b568568-55de-4e4c-83c8-6ba1e15cbc39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e11f68a514564dc4b03c5c1c21c13e4f', 'request_id': 'req-2b568568-55de-4e4c-83c8-6ba1e15cbc39'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:50:06.014712 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-2b568568-55de-4e4c-83c8-6ba1e15cbc39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e11f68a514564dc4b03c5c1c21c13e4f', 'request_id': 'req-2b568568-55de-4e4c-83c8-6ba1e15cbc39'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:50:06.026958 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-2b568568-55de-4e4c-83c8-6ba1e15cbc39 admin admin] The token cache is being invalidate because project e11f68a514564dc4b03c5c1c21c13e4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:50:06.027252 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-2b568568-55de-4e4c-83c8-6ba1e15cbc39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:50:06.027907 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-2b568568-55de-4e4c-83c8-6ba1e15cbc39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:50:06.029521 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1440/2870] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Aug 6 01:50:05 2022] DELETE /identity/v3/projects/e11f68a514564dc4b03c5c1c21c13e4f => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:50:06.034420 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c39d65a-ce98-4fc4-bca6-51ef68362823 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:50:06.034719 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c39d65a-ce98-4fc4-bca6-51ef68362823 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:50:06.035005 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c39d65a-ce98-4fc4-bca6-51ef68362823 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:50:06.055182 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-0c39d65a-ce98-4fc4-bca6-51ef68362823 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:50:06.056493 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-0c39d65a-ce98-4fc4-bca6-51ef68362823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:50:06.115844 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1431/2871] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:50:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:50:06.124394 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b7238b-c230-4a41-85a7-a5a67352c5a2 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:50:06.124690 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b7238b-c230-4a41-85a7-a5a67352c5a2 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:50:06.124999 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b7238b-c230-4a41-85a7-a5a67352c5a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:50:06.145199 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-a0b7238b-c230-4a41-85a7-a5a67352c5a2 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:50:06.146411 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-a0b7238b-c230-4a41-85a7-a5a67352c5a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:50:06.154451 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1441/2872] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:50:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:50:06.158652 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c39d65a-ce98-4fc4-bca6-51ef68362823 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:50:06.168282 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c39d65a-ce98-4fc4-bca6-51ef68362823 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:50:06.172937 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a25a24-a821-45d9-b30c-49e0500df24f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:50:06.173602 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a25a24-a821-45d9-b30c-49e0500df24f admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:50:06.173888 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a25a24-a821-45d9-b30c-49e0500df24f admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:50:06.174166 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a25a24-a821-45d9-b30c-49e0500df24f admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:50:06.174863 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a25a24-a821-45d9-b30c-49e0500df24f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:50:06.175173 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a25a24-a821-45d9-b30c-49e0500df24f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:50:06.176755 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a25a24-a821-45d9-b30c-49e0500df24f admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:50:06.188695 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1432/2873] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Aug 6 01:50:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:50:06.193089 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b7238b-c230-4a41-85a7-a5a67352c5a2 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:50:06.203816 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b7238b-c230-4a41-85a7-a5a67352c5a2 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:50:06.208654 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f482b997-1f9c-460a-bd68-e43882714ae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:50:06.209291 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f482b997-1f9c-460a-bd68-e43882714ae3 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:50:06.209588 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f482b997-1f9c-460a-bd68-e43882714ae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:50:06.209858 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f482b997-1f9c-460a-bd68-e43882714ae3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:50:06.210539 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f482b997-1f9c-460a-bd68-e43882714ae3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:50:06.212845 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f482b997-1f9c-460a-bd68-e43882714ae3 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:50:06.242898 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1442/2874] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Aug 6 01:50:06 2022] POST /identity/v3/projects => generated 375 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:50:06.247119 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a25a24-a821-45d9-b30c-49e0500df24f admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:50:06.259214 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a25a24-a821-45d9-b30c-49e0500df24f admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:50:06.323001 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d2f6d7-4a71-4135-9bda-7a40bb2480ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:50:06.323705 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d2f6d7-4a71-4135-9bda-7a40bb2480ec admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:50:06.323986 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d2f6d7-4a71-4135-9bda-7a40bb2480ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:50:06.324261 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d2f6d7-4a71-4135-9bda-7a40bb2480ec admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:50:06.324949 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d2f6d7-4a71-4135-9bda-7a40bb2480ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:50:06.326893 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d2f6d7-4a71-4135-9bda-7a40bb2480ec admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:50:06.354380 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1433/2875] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:50:06 2022] POST /identity/v3/users => generated 288 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:50:06.358874 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f482b997-1f9c-460a-bd68-e43882714ae3 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:50:06.370509 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f482b997-1f9c-460a-bd68-e43882714ae3 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:50:06.375760 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e3e959-7aee-415a-8007-03d59b541145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:50:06.376370 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e3e959-7aee-415a-8007-03d59b541145 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:50:06.376647 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e3e959-7aee-415a-8007-03d59b541145 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:50:06.376916 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e3e959-7aee-415a-8007-03d59b541145 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:50:06.377638 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e3e959-7aee-415a-8007-03d59b541145 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:50:06.379210 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e3e959-7aee-415a-8007-03d59b541145 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:50:06.389114 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1443/2876] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:50:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:50:06.393384 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d2f6d7-4a71-4135-9bda-7a40bb2480ec admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:50:06.403560 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d2f6d7-4a71-4135-9bda-7a40bb2480ec admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:50:06.408229 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c815a7e-7233-4c06-8025-e2007b4a04f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:50:06.408842 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c815a7e-7233-4c06-8025-e2007b4a04f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:50:06.409158 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c815a7e-7233-4c06-8025-e2007b4a04f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:50:06.409436 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c815a7e-7233-4c06-8025-e2007b4a04f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:50:06.410124 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c815a7e-7233-4c06-8025-e2007b4a04f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:50:06.411702 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c815a7e-7233-4c06-8025-e2007b4a04f1 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:50:06.423020 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1434/2877] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:50:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 01:50:06.427242 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e3e959-7aee-415a-8007-03d59b541145 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:50:06.437818 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e3e959-7aee-415a-8007-03d59b541145 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:50:06.442361 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68f65e1-0669-4043-8f7b-5a4bd07fcc5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:50:06.443013 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68f65e1-0669-4043-8f7b-5a4bd07fcc5d admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:50:06.443293 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68f65e1-0669-4043-8f7b-5a4bd07fcc5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:50:06.443567 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68f65e1-0669-4043-8f7b-5a4bd07fcc5d admin admin] PATH_INFO: `/v3/projects/2dd6a14f8dbb4693b3a8483447032608/users/30370badf8354aa68b5e060b969193a9/roles/61b3b93d37a047e4bb8369b46fe20db9` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:50:06.458331 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68f65e1-0669-4043-8f7b-5a4bd07fcc5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2dd6a14f8dbb4693b3a8483447032608, user_id=30370badf8354aa68b5e060b969193a9, role_id=61b3b93d37a047e4bb8369b46fe20db9)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:50:06.461600 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68f65e1-0669-4043-8f7b-5a4bd07fcc5d admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:50:06.474037 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1444/2878] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:50:06 2022] PUT /identity/v3/projects/2dd6a14f8dbb4693b3a8483447032608/users/30370badf8354aa68b5e060b969193a9/roles/61b3b93d37a047e4bb8369b46fe20db9 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:50:06.485601 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91be5dc-8cbc-4e80-84c4-7693b6ea4bef None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:50:06.485909 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91be5dc-8cbc-4e80-84c4-7693b6ea4bef None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:50:06.486170 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91be5dc-8cbc-4e80-84c4-7693b6ea4bef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:50:06.514685 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-f91be5dc-8cbc-4e80-84c4-7693b6ea4bef None None] MFA Rules not processed for user `30370badf8354aa68b5e060b969193a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:50:06.515839 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-f91be5dc-8cbc-4e80-84c4-7693b6ea4bef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:50:06.558835 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:50:06.583495 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1435/2879] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:50:06 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:50:06.596327 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68f65e1-0669-4043-8f7b-5a4bd07fcc5d admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:50:06.598077 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-e68f65e1-0669-4043-8f7b-5a4bd07fcc5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:50:06.645256 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:50:06.682868 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68f65e1-0669-4043-8f7b-5a4bd07fcc5d admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:50:06.686586 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a479089e-4865-4da1-b1f8-9cd8fd32d258 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abb76a5c8ef74bf2b18c56074daaad58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:50:06.687207 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a479089e-4865-4da1-b1f8-9cd8fd32d258 service glance] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:50:06.687491 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a479089e-4865-4da1-b1f8-9cd8fd32d258 service glance] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:50:06.687771 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a479089e-4865-4da1-b1f8-9cd8fd32d258 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:50:06.696893 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a479089e-4865-4da1-b1f8-9cd8fd32d258 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:50:06.698362 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a479089e-4865-4da1-b1f8-9cd8fd32d258 service glance] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:50:06.707147 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1445/2880] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:50:06 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:50:07.106513 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91be5dc-8cbc-4e80-84c4-7693b6ea4bef None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:50:07.107978 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-f91be5dc-8cbc-4e80-84c4-7693b6ea4bef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:50:07.171730 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91be5dc-8cbc-4e80-84c4-7693b6ea4bef None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:50:07.176202 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d27730-f1a9-40f6-8dac-a4963c086068 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1be1e6173e374a60b9e1fbf51a1ab571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:50:07.176988 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d27730-f1a9-40f6-8dac-a4963c086068 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:50:07.177433 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d27730-f1a9-40f6-8dac-a4963c086068 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:50:07.177790 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d27730-f1a9-40f6-8dac-a4963c086068 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:50:07.179981 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-d1d27730-f1a9-40f6-8dac-a4963c086068 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:50:07.239594 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d27730-f1a9-40f6-8dac-a4963c086068 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:50:07.241026 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d27730-f1a9-40f6-8dac-a4963c086068 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:50:07.250298 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1436/2881] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:50:07 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:50:16.703183 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=16 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:50:17.245907 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=56 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:50:19.480675 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db901be6-7a09-4a59-b6f9-8ededf36b033 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:50:19.481246 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db901be6-7a09-4a59-b6f9-8ededf36b033 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:50:19.481246 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db901be6-7a09-4a59-b6f9-8ededf36b033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:50:19.499767 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-db901be6-7a09-4a59-b6f9-8ededf36b033 None None] MFA Rules not processed for user `6ac05cb43bb146cbb0a548ff3148a77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:50:19.500961 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-db901be6-7a09-4a59-b6f9-8ededf36b033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:50:19.508058 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1446/2882] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:50:19 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:50:20.259692 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d27730-f1a9-40f6-8dac-a4963c086068 service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:50:20.261292 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-d1d27730-f1a9-40f6-8dac-a4963c086068 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:50:20.332737 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d27730-f1a9-40f6-8dac-a4963c086068 service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:50:20.338089 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8446ac-5a54-4875-8b5f-e51508388402 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa3b93ddd9464a2cba724d67b56b477a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:50:20.338718 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8446ac-5a54-4875-8b5f-e51508388402 service placement] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:50:20.339006 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8446ac-5a54-4875-8b5f-e51508388402 service placement] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:50:20.339287 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8446ac-5a54-4875-8b5f-e51508388402 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:50:20.341070 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-ff8446ac-5a54-4875-8b5f-e51508388402 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:50:20.352895 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8446ac-5a54-4875-8b5f-e51508388402 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:50:20.354542 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8446ac-5a54-4875-8b5f-e51508388402 service placement] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:50:20.364547 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1437/2883] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:50:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:50:22.855253 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db901be6-7a09-4a59-b6f9-8ededf36b033 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:50:22.856933 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-db901be6-7a09-4a59-b6f9-8ededf36b033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:50:22.944110 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db901be6-7a09-4a59-b6f9-8ededf36b033 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:50:22.951597 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf01408c-2128-40dd-9194-dc6c3a6e3ead service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87adc1d5bc240e99f95f6595b350be4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:50:22.952227 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf01408c-2128-40dd-9194-dc6c3a6e3ead service nova] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:50:22.952510 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf01408c-2128-40dd-9194-dc6c3a6e3ead service nova] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:50:22.952787 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf01408c-2128-40dd-9194-dc6c3a6e3ead service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:50:22.954638 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-cf01408c-2128-40dd-9194-dc6c3a6e3ead service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:50:22.967328 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf01408c-2128-40dd-9194-dc6c3a6e3ead service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:50:22.968891 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf01408c-2128-40dd-9194-dc6c3a6e3ead service nova] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:50:22.979950 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1447/2884] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:50:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:50:30.360260 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=32 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:50:30.636012 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8446ac-5a54-4875-8b5f-e51508388402 service placement] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:50:30.704705 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8446ac-5a54-4875-8b5f-e51508388402 service placement] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:50:30.709314 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597edb88-b03f-4be1-ae05-99919c8d805b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:50:30.709949 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597edb88-b03f-4be1-ae05-99919c8d805b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:50:30.710233 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597edb88-b03f-4be1-ae05-99919c8d805b admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:50:30.711120 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597edb88-b03f-4be1-ae05-99919c8d805b admin admin] PATH_INFO: `/v3/users/30370badf8354aa68b5e060b969193a9` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:50:30.712305 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597edb88-b03f-4be1-ae05-99919c8d805b admin admin] RBAC: Authorizing `identity:delete_user(user_id=30370badf8354aa68b5e060b969193a9)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:50:30.714270 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597edb88-b03f-4be1-ae05-99919c8d805b admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:50:30.759756 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-597edb88-b03f-4be1-ae05-99919c8d805b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30370badf8354aa68b5e060b969193a9'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:50:30.760435 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-597edb88-b03f-4be1-ae05-99919c8d805b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30370badf8354aa68b5e060b969193a9'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:50:30.761081 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-597edb88-b03f-4be1-ae05-99919c8d805b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30370badf8354aa68b5e060b969193a9'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:50:30.774631 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-597edb88-b03f-4be1-ae05-99919c8d805b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30370badf8354aa68b5e060b969193a9'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:50:30.782400 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-597edb88-b03f-4be1-ae05-99919c8d805b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30370badf8354aa68b5e060b969193a9'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:50:30.796443 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1438/2885] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Aug 6 01:50:30 2022] DELETE /identity/v3/users/30370badf8354aa68b5e060b969193a9 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:50:30.801456 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c532c65b-fbf0-41f6-a6f1-5ce6a89c73ff None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:50:30.801957 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c532c65b-fbf0-41f6-a6f1-5ce6a89c73ff None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:50:30.802199 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c532c65b-fbf0-41f6-a6f1-5ce6a89c73ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:50:30.821923 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-c532c65b-fbf0-41f6-a6f1-5ce6a89c73ff None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:50:30.823132 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-c532c65b-fbf0-41f6-a6f1-5ce6a89c73ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:50:30.884531 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1448/2886] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:50:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:50:30.893967 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597edb88-b03f-4be1-ae05-99919c8d805b admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:50:30.895391 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-597edb88-b03f-4be1-ae05-99919c8d805b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:50:30.955221 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597edb88-b03f-4be1-ae05-99919c8d805b admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:50:30.959602 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6645485-e699-4646-9fba-6be6e09e09e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:50:30.960251 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6645485-e699-4646-9fba-6be6e09e09e4 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:50:30.961210 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6645485-e699-4646-9fba-6be6e09e09e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:50:30.961554 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6645485-e699-4646-9fba-6be6e09e09e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:50:30.970775 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6645485-e699-4646-9fba-6be6e09e09e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:50:30.972196 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6645485-e699-4646-9fba-6be6e09e09e4 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:50:30.982538 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1439/2887] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:50:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:50:31.256974 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c532c65b-fbf0-41f6-a6f1-5ce6a89c73ff None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:50:31.259119 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-c532c65b-fbf0-41f6-a6f1-5ce6a89c73ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:50:31.274708 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c532c65b-fbf0-41f6-a6f1-5ce6a89c73ff None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:50:31.279507 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79956fc5-608f-4d0e-96fa-65039071371d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:50:31.280795 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79956fc5-608f-4d0e-96fa-65039071371d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:50:31.281111 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79956fc5-608f-4d0e-96fa-65039071371d admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:50:31.281369 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79956fc5-608f-4d0e-96fa-65039071371d admin admin] PATH_INFO: `/v3/projects/2dd6a14f8dbb4693b3a8483447032608` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:50:31.282585 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79956fc5-608f-4d0e-96fa-65039071371d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2dd6a14f8dbb4693b3a8483447032608)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:50:31.284428 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79956fc5-608f-4d0e-96fa-65039071371d admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:50:31.331557 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-79956fc5-608f-4d0e-96fa-65039071371d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2dd6a14f8dbb4693b3a8483447032608', 'request_id': 'req-79956fc5-608f-4d0e-96fa-65039071371d'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:50:31.338785 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-79956fc5-608f-4d0e-96fa-65039071371d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2dd6a14f8dbb4693b3a8483447032608', 'request_id': 'req-79956fc5-608f-4d0e-96fa-65039071371d'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:50:31.344528 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-79956fc5-608f-4d0e-96fa-65039071371d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2dd6a14f8dbb4693b3a8483447032608', 'request_id': 'req-79956fc5-608f-4d0e-96fa-65039071371d'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:50:31.358485 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-79956fc5-608f-4d0e-96fa-65039071371d admin admin] The token cache is being invalidate because project 2dd6a14f8dbb4693b3a8483447032608 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:50:31.358782 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-79956fc5-608f-4d0e-96fa-65039071371d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:50:31.359403 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-79956fc5-608f-4d0e-96fa-65039071371d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:50:31.360899 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1449/2888] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Aug 6 01:50:31 2022] DELETE /identity/v3/projects/2dd6a14f8dbb4693b3a8483447032608 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:50:31.366002 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b4de4a-239d-494f-b074-f2369b95f2b6 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:50:31.366489 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b4de4a-239d-494f-b074-f2369b95f2b6 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:50:31.367181 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b4de4a-239d-494f-b074-f2369b95f2b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:50:31.387529 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-10b4de4a-239d-494f-b074-f2369b95f2b6 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:50:31.388672 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-10b4de4a-239d-494f-b074-f2369b95f2b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:50:31.406201 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:50:31.450282 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1440/2889] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:50:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:50:31.458092 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80210fe-e8a5-48b6-9649-0fbbcbe8e499 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:50:31.458378 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80210fe-e8a5-48b6-9649-0fbbcbe8e499 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:50:31.458648 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80210fe-e8a5-48b6-9649-0fbbcbe8e499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:50:31.466433 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:50:31.480528 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-f80210fe-e8a5-48b6-9649-0fbbcbe8e499 None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:50:31.481833 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-f80210fe-e8a5-48b6-9649-0fbbcbe8e499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:50:31.490033 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1450/2890] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:50:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:50:31.494939 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b4de4a-239d-494f-b074-f2369b95f2b6 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:50:31.504813 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b4de4a-239d-494f-b074-f2369b95f2b6 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:50:31.510158 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a186545-651a-4552-9f26-a6ea0266d3c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:50:31.510759 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a186545-651a-4552-9f26-a6ea0266d3c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:50:31.511041 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a186545-651a-4552-9f26-a6ea0266d3c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:50:31.511317 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a186545-651a-4552-9f26-a6ea0266d3c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:50:31.512007 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a186545-651a-4552-9f26-a6ea0266d3c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:50:31.512312 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a186545-651a-4552-9f26-a6ea0266d3c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:50:31.513925 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a186545-651a-4552-9f26-a6ea0266d3c9 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:50:31.525435 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1441/2891] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Aug 6 01:50:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:50:31.529631 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80210fe-e8a5-48b6-9649-0fbbcbe8e499 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:50:31.538854 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80210fe-e8a5-48b6-9649-0fbbcbe8e499 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:50:31.543478 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893401bf-2359-4623-86ec-82fd4d2d0463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:50:31.544087 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893401bf-2359-4623-86ec-82fd4d2d0463 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:50:31.544362 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893401bf-2359-4623-86ec-82fd4d2d0463 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:50:31.544632 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893401bf-2359-4623-86ec-82fd4d2d0463 admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:50:31.545312 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893401bf-2359-4623-86ec-82fd4d2d0463 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:50:31.547212 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893401bf-2359-4623-86ec-82fd4d2d0463 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:50:31.575219 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1451/2892] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Aug 6 01:50:31 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:50:31.579559 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a186545-651a-4552-9f26-a6ea0266d3c9 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:50:31.591500 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a186545-651a-4552-9f26-a6ea0266d3c9 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:50:31.648759 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7637a8a-55a8-4794-8086-ea0dc9f28647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:50:31.649476 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7637a8a-55a8-4794-8086-ea0dc9f28647 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:50:31.649763 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7637a8a-55a8-4794-8086-ea0dc9f28647 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:50:31.650044 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7637a8a-55a8-4794-8086-ea0dc9f28647 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:50:31.650722 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7637a8a-55a8-4794-8086-ea0dc9f28647 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:50:31.652635 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7637a8a-55a8-4794-8086-ea0dc9f28647 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:50:31.673423 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1442/2893] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Aug 6 01:50:31 2022] POST /identity/v3/users => generated 292 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:50:31.677720 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893401bf-2359-4623-86ec-82fd4d2d0463 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:50:31.687891 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893401bf-2359-4623-86ec-82fd4d2d0463 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:50:31.692525 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400b2f53-b512-4b7f-b58d-760478751f7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:50:31.693154 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400b2f53-b512-4b7f-b58d-760478751f7a admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:50:31.693440 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400b2f53-b512-4b7f-b58d-760478751f7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:50:31.693717 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400b2f53-b512-4b7f-b58d-760478751f7a admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:50:31.694416 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400b2f53-b512-4b7f-b58d-760478751f7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:50:31.696129 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400b2f53-b512-4b7f-b58d-760478751f7a admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:50:31.705731 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1452/2894] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:50:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:50:31.709588 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7637a8a-55a8-4794-8086-ea0dc9f28647 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:50:31.720233 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7637a8a-55a8-4794-8086-ea0dc9f28647 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:50:31.725006 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3f9806-819c-4f3c-aea9-6ad4a14a97e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:50:31.725791 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3f9806-819c-4f3c-aea9-6ad4a14a97e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:50:31.726736 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3f9806-819c-4f3c-aea9-6ad4a14a97e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:50:31.727087 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3f9806-819c-4f3c-aea9-6ad4a14a97e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:50:31.727769 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3f9806-819c-4f3c-aea9-6ad4a14a97e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:50:31.729388 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3f9806-819c-4f3c-aea9-6ad4a14a97e3 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:50:31.740325 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1443/2895] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Aug 6 01:50:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 01:50:31.744391 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400b2f53-b512-4b7f-b58d-760478751f7a admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:50:31.754413 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400b2f53-b512-4b7f-b58d-760478751f7a admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:50:31.759561 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cde854-638d-4495-90f2-4dc7538af0ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:50:31.760211 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cde854-638d-4495-90f2-4dc7538af0ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:50:31.760492 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cde854-638d-4495-90f2-4dc7538af0ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:50:31.760764 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cde854-638d-4495-90f2-4dc7538af0ff admin admin] PATH_INFO: `/v3/projects/8b5248618fa94fdca5b220700656613f/users/2c0ad3576abb481baeec268f14f7f263/roles/61b3b93d37a047e4bb8369b46fe20db9` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:50:31.775818 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cde854-638d-4495-90f2-4dc7538af0ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b5248618fa94fdca5b220700656613f, user_id=2c0ad3576abb481baeec268f14f7f263, role_id=61b3b93d37a047e4bb8369b46fe20db9)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:50:31.778504 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cde854-638d-4495-90f2-4dc7538af0ff admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:50:31.792097 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1453/2896] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Aug 6 01:50:31 2022] PUT /identity/v3/projects/8b5248618fa94fdca5b220700656613f/users/2c0ad3576abb481baeec268f14f7f263/roles/61b3b93d37a047e4bb8369b46fe20db9 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:50:31.803540 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16934aa-758d-4033-ae74-414228729a54 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:50:31.803829 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16934aa-758d-4033-ae74-414228729a54 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:50:31.804123 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16934aa-758d-4033-ae74-414228729a54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:50:31.831576 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-b16934aa-758d-4033-ae74-414228729a54 None None] MFA Rules not processed for user `2c0ad3576abb481baeec268f14f7f263`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:50:31.832770 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-b16934aa-758d-4033-ae74-414228729a54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:50:31.897105 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1444/2897] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:50:31 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:50:31.909139 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cde854-638d-4495-90f2-4dc7538af0ff admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:50:31.910803 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-b0cde854-638d-4495-90f2-4dc7538af0ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:50:31.973457 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cde854-638d-4495-90f2-4dc7538af0ff admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:50:31.977803 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147ac270-d867-4a1b-a5c7-b97d499fcce6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1be1e6173e374a60b9e1fbf51a1ab571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:50:31.978449 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147ac270-d867-4a1b-a5c7-b97d499fcce6 service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:50:31.978727 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147ac270-d867-4a1b-a5c7-b97d499fcce6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:50:31.979000 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147ac270-d867-4a1b-a5c7-b97d499fcce6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:50:31.987361 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147ac270-d867-4a1b-a5c7-b97d499fcce6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:50:31.988768 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147ac270-d867-4a1b-a5c7-b97d499fcce6 service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:50:31.997961 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1454/2898] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:50:31 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:50:34.551766 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16934aa-758d-4033-ae74-414228729a54 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:50:34.572792 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:50:34.621106 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16934aa-758d-4033-ae74-414228729a54 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:50:34.626255 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340b9846-9408-457c-a6ce-2ffb7687a19c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:50:34.626943 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340b9846-9408-457c-a6ce-2ffb7687a19c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:50:34.627271 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340b9846-9408-457c-a6ce-2ffb7687a19c admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:50:34.627579 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340b9846-9408-457c-a6ce-2ffb7687a19c admin admin] PATH_INFO: `/v3/users/2c0ad3576abb481baeec268f14f7f263` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:50:34.628757 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340b9846-9408-457c-a6ce-2ffb7687a19c admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c0ad3576abb481baeec268f14f7f263)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:50:34.630750 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340b9846-9408-457c-a6ce-2ffb7687a19c admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:50:34.685971 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-340b9846-9408-457c-a6ce-2ffb7687a19c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c0ad3576abb481baeec268f14f7f263'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:50:34.686306 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-340b9846-9408-457c-a6ce-2ffb7687a19c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c0ad3576abb481baeec268f14f7f263'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:50:34.686825 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-340b9846-9408-457c-a6ce-2ffb7687a19c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c0ad3576abb481baeec268f14f7f263'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:50:34.700404 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-340b9846-9408-457c-a6ce-2ffb7687a19c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c0ad3576abb481baeec268f14f7f263'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:50:34.702934 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147ac270-d867-4a1b-a5c7-b97d499fcce6 service cinder] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:50:34.705471 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-147ac270-d867-4a1b-a5c7-b97d499fcce6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:50:34.708600 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-340b9846-9408-457c-a6ce-2ffb7687a19c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c0ad3576abb481baeec268f14f7f263'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:50:34.727747 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1445/2899] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Aug 6 01:50:34 2022] DELETE /identity/v3/users/2c0ad3576abb481baeec268f14f7f263 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:50:34.732351 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0242c3c4-18a0-4f2d-a1b5-c9bbc88151fc None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:50:34.733035 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0242c3c4-18a0-4f2d-a1b5-c9bbc88151fc None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:50:34.733478 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0242c3c4-18a0-4f2d-a1b5-c9bbc88151fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:50:34.758991 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-0242c3c4-18a0-4f2d-a1b5-c9bbc88151fc None None] MFA Rules not processed for user `ca5d3774b93a41d1a8b95796b7899d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:50:34.759869 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-0242c3c4-18a0-4f2d-a1b5-c9bbc88151fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:50:34.796404 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147ac270-d867-4a1b-a5c7-b97d499fcce6 service cinder] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:50:34.821746 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1446/2900] 10.222.0.26 () {58 vars in 958 bytes} [Sat Aug 6 01:50:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:50:34.831207 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0242c3c4-18a0-4f2d-a1b5-c9bbc88151fc None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:50:34.832731 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-0242c3c4-18a0-4f2d-a1b5-c9bbc88151fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:50:34.834601 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:50:34.863252 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94404f70-127e-43ae-9fbd-0f6cfc02a12b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87adc1d5bc240e99f95f6595b350be4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:50:34.863873 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94404f70-127e-43ae-9fbd-0f6cfc02a12b service nova] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:50:34.864156 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94404f70-127e-43ae-9fbd-0f6cfc02a12b service nova] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:50:34.864430 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94404f70-127e-43ae-9fbd-0f6cfc02a12b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:50:34.866678 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-94404f70-127e-43ae-9fbd-0f6cfc02a12b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:50:34.875604 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:50:34.879374 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94404f70-127e-43ae-9fbd-0f6cfc02a12b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:50:34.883301 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94404f70-127e-43ae-9fbd-0f6cfc02a12b service nova] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:50:34.893997 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0242c3c4-18a0-4f2d-a1b5-c9bbc88151fc None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:50:34.894440 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1455/2901] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:50:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 01:50:34.899000 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc901a21-4b6a-4c89-9cdb-aa428cd2f522 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:50:34.899350 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc901a21-4b6a-4c89-9cdb-aa428cd2f522 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:50:34.899633 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc901a21-4b6a-4c89-9cdb-aa428cd2f522 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:50:34.899912 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc901a21-4b6a-4c89-9cdb-aa428cd2f522 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:50:34.909734 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc901a21-4b6a-4c89-9cdb-aa428cd2f522 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:50:34.911166 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc901a21-4b6a-4c89-9cdb-aa428cd2f522 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:50:34.921605 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1447/2902] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:50:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 01:50:35.243242 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94404f70-127e-43ae-9fbd-0f6cfc02a12b service nova] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:50:35.244905 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-94404f70-127e-43ae-9fbd-0f6cfc02a12b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:50:35.258657 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94404f70-127e-43ae-9fbd-0f6cfc02a12b service nova] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:50:35.263322 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7f1986-b8c0-48d1-aeba-9ba17b6b1f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:50:35.263943 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7f1986-b8c0-48d1-aeba-9ba17b6b1f60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:50:35.264228 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7f1986-b8c0-48d1-aeba-9ba17b6b1f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:50:35.264513 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7f1986-b8c0-48d1-aeba-9ba17b6b1f60 admin admin] PATH_INFO: `/v3/projects/8b5248618fa94fdca5b220700656613f` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:50:35.265684 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7f1986-b8c0-48d1-aeba-9ba17b6b1f60 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b5248618fa94fdca5b220700656613f)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:50:35.267603 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7f1986-b8c0-48d1-aeba-9ba17b6b1f60 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:50:35.317459 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-cb7f1986-b8c0-48d1-aeba-9ba17b6b1f60 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b5248618fa94fdca5b220700656613f', 'request_id': 'req-cb7f1986-b8c0-48d1-aeba-9ba17b6b1f60'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:50:35.324004 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-cb7f1986-b8c0-48d1-aeba-9ba17b6b1f60 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b5248618fa94fdca5b220700656613f', 'request_id': 'req-cb7f1986-b8c0-48d1-aeba-9ba17b6b1f60'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:50:35.329390 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-cb7f1986-b8c0-48d1-aeba-9ba17b6b1f60 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b5248618fa94fdca5b220700656613f', 'request_id': 'req-cb7f1986-b8c0-48d1-aeba-9ba17b6b1f60'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:50:35.342505 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-cb7f1986-b8c0-48d1-aeba-9ba17b6b1f60 admin admin] The token cache is being invalidate because project 8b5248618fa94fdca5b220700656613f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:50:35.342798 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-cb7f1986-b8c0-48d1-aeba-9ba17b6b1f60 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:50:35.343425 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-cb7f1986-b8c0-48d1-aeba-9ba17b6b1f60 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:50:35.345025 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1456/2903] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Aug 6 01:50:35 2022] DELETE /identity/v3/projects/8b5248618fa94fdca5b220700656613f => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:50:36.997376 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc901a21-4b6a-4c89-9cdb-aa428cd2f522 service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:50:36.999127 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-dc901a21-4b6a-4c89-9cdb-aa428cd2f522 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:50:37.011341 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc901a21-4b6a-4c89-9cdb-aa428cd2f522 service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:50:37.060798 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d82e55e-c1de-43b7-88fd-1f5010dc6189 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:50:37.061445 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d82e55e-c1de-43b7-88fd-1f5010dc6189 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:50:37.061727 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d82e55e-c1de-43b7-88fd-1f5010dc6189 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:50:37.062007 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d82e55e-c1de-43b7-88fd-1f5010dc6189 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:50:37.063786 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-1d82e55e-c1de-43b7-88fd-1f5010dc6189 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:50:37.125689 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d82e55e-c1de-43b7-88fd-1f5010dc6189 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:50:37.127153 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d82e55e-c1de-43b7-88fd-1f5010dc6189 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:50:37.136974 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1448/2904] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:50:36 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:50:45.339479 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=43,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:50:47.132786 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=59 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:52:09.968442 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7f1986-b8c0-48d1-aeba-9ba17b6b1f60 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:52:09.969927 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-cb7f1986-b8c0-48d1-aeba-9ba17b6b1f60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:52:10.107442 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7f1986-b8c0-48d1-aeba-9ba17b6b1f60 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:52:10.112708 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0f87eb-1db2-4290-bad1-df63b32317c6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa3b93ddd9464a2cba724d67b56b477a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:52:10.113368 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0f87eb-1db2-4290-bad1-df63b32317c6 service placement] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:52:10.113650 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0f87eb-1db2-4290-bad1-df63b32317c6 service placement] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:52:10.113921 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0f87eb-1db2-4290-bad1-df63b32317c6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:52:10.115646 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-7c0f87eb-1db2-4290-bad1-df63b32317c6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:52:10.127291 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0f87eb-1db2-4290-bad1-df63b32317c6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:52:10.128861 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0f87eb-1db2-4290-bad1-df63b32317c6 service placement] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:52:10.139582 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1457/2905] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:52:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:52:20.134964 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=63 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:52:43.179459 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d82e55e-c1de-43b7-88fd-1f5010dc6189 service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:52:43.181031 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-1d82e55e-c1de-43b7-88fd-1f5010dc6189 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:52:43.365941 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d82e55e-c1de-43b7-88fd-1f5010dc6189 service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:52:43.374389 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0355aec-98fc-4e76-b671-d14b1ecb82d6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87adc1d5bc240e99f95f6595b350be4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:52:43.374389 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0355aec-98fc-4e76-b671-d14b1ecb82d6 service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:52:43.374389 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0355aec-98fc-4e76-b671-d14b1ecb82d6 service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:52:43.374389 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0355aec-98fc-4e76-b671-d14b1ecb82d6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:52:43.375572 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-c0355aec-98fc-4e76-b671-d14b1ecb82d6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:52:43.388986 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0355aec-98fc-4e76-b671-d14b1ecb82d6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:52:43.390619 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0355aec-98fc-4e76-b671-d14b1ecb82d6 service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:52:43.405572 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1449/2906] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:52:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 01:52:53.399462 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=68 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:52:55.500389 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0f87eb-1db2-4290-bad1-df63b32317c6 service placement] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:52:55.502027 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-7c0f87eb-1db2-4290-bad1-df63b32317c6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:52:55.536510 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0f87eb-1db2-4290-bad1-df63b32317c6 service placement] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:52:55.540161 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6b576e-5809-4078-9493-c6701213391a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ac0e31c84b4445fa167bbe4954066c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:52:55.540777 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6b576e-5809-4078-9493-c6701213391a service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:52:55.541086 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6b576e-5809-4078-9493-c6701213391a service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:52:55.541372 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6b576e-5809-4078-9493-c6701213391a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:52:55.543838 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-1d6b576e-5809-4078-9493-c6701213391a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:52:55.629079 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6b576e-5809-4078-9493-c6701213391a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:52:55.630516 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6b576e-5809-4078-9493-c6701213391a service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:52:55.649037 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1458/2907] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:52:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:53:01.852733 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0355aec-98fc-4e76-b671-d14b1ecb82d6 service nova] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:53:01.854306 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-c0355aec-98fc-4e76-b671-d14b1ecb82d6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:53:01.871944 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0355aec-98fc-4e76-b671-d14b1ecb82d6 service nova] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:53:01.880015 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e49627-82e8-4572-bbbe-093b76dd65a5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa3b93ddd9464a2cba724d67b56b477a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:53:01.880591 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e49627-82e8-4572-bbbe-093b76dd65a5 service placement] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:53:01.880877 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e49627-82e8-4572-bbbe-093b76dd65a5 service placement] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:53:01.881189 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e49627-82e8-4572-bbbe-093b76dd65a5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:53:01.883078 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-e5e49627-82e8-4572-bbbe-093b76dd65a5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:53:01.897752 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e49627-82e8-4572-bbbe-093b76dd65a5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:53:01.899387 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e49627-82e8-4572-bbbe-093b76dd65a5 service placement] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:53:01.912409 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1450/2908] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:53:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:53:01.926221 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6b576e-5809-4078-9493-c6701213391a service neutron] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:53:01.927752 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-1d6b576e-5809-4078-9493-c6701213391a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:53:01.962967 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6b576e-5809-4078-9493-c6701213391a service neutron] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:53:01.973346 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b367e91-eb31-4ac6-b36b-0fac42d588f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:53:01.974003 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b367e91-eb31-4ac6-b36b-0fac42d588f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:53:01.974466 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b367e91-eb31-4ac6-b36b-0fac42d588f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:53:01.974743 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b367e91-eb31-4ac6-b36b-0fac42d588f5 admin admin] PATH_INFO: `/v3/users/6ac05cb43bb146cbb0a548ff3148a77a` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:53:01.975942 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b367e91-eb31-4ac6-b36b-0fac42d588f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ac05cb43bb146cbb0a548ff3148a77a)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:53:01.977940 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b367e91-eb31-4ac6-b36b-0fac42d588f5 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:53:02.028325 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-2b367e91-eb31-4ac6-b36b-0fac42d588f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ac05cb43bb146cbb0a548ff3148a77a'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:53:02.035674 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-2b367e91-eb31-4ac6-b36b-0fac42d588f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ac05cb43bb146cbb0a548ff3148a77a'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:53:02.036361 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-2b367e91-eb31-4ac6-b36b-0fac42d588f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ac05cb43bb146cbb0a548ff3148a77a'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:53:02.036962 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-2b367e91-eb31-4ac6-b36b-0fac42d588f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ac05cb43bb146cbb0a548ff3148a77a'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:53:02.050880 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-2b367e91-eb31-4ac6-b36b-0fac42d588f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ac05cb43bb146cbb0a548ff3148a77a'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:53:02.067428 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1459/2909] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Aug 6 01:53:01 2022] DELETE /identity/v3/users/6ac05cb43bb146cbb0a548ff3148a77a => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:53:02.243460 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e49627-82e8-4572-bbbe-093b76dd65a5 service placement] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:53:02.256116 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e49627-82e8-4572-bbbe-093b76dd65a5 service placement] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:53:02.314258 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c1dfea-12a6-403c-9511-3a2c5f6c5ad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:53:02.314834 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c1dfea-12a6-403c-9511-3a2c5f6c5ad0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:53:02.315121 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c1dfea-12a6-403c-9511-3a2c5f6c5ad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:53:02.315405 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c1dfea-12a6-403c-9511-3a2c5f6c5ad0 admin admin] PATH_INFO: `/v3/projects/3bea2c3f927a4bccb59065a8380b98c9` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:53:02.316566 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c1dfea-12a6-403c-9511-3a2c5f6c5ad0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3bea2c3f927a4bccb59065a8380b98c9)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:53:02.318513 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c1dfea-12a6-403c-9511-3a2c5f6c5ad0 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:53:02.374535 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-62c1dfea-12a6-403c-9511-3a2c5f6c5ad0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3bea2c3f927a4bccb59065a8380b98c9', 'request_id': 'req-62c1dfea-12a6-403c-9511-3a2c5f6c5ad0'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:53:02.380644 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-62c1dfea-12a6-403c-9511-3a2c5f6c5ad0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3bea2c3f927a4bccb59065a8380b98c9', 'request_id': 'req-62c1dfea-12a6-403c-9511-3a2c5f6c5ad0'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:53:02.386280 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-62c1dfea-12a6-403c-9511-3a2c5f6c5ad0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3bea2c3f927a4bccb59065a8380b98c9', 'request_id': 'req-62c1dfea-12a6-403c-9511-3a2c5f6c5ad0'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:53:02.399429 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-62c1dfea-12a6-403c-9511-3a2c5f6c5ad0 admin admin] The token cache is being invalidate because project 3bea2c3f927a4bccb59065a8380b98c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:53:02.399726 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-62c1dfea-12a6-403c-9511-3a2c5f6c5ad0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:53:02.400357 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-62c1dfea-12a6-403c-9511-3a2c5f6c5ad0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:53:02.401924 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1451/2910] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Aug 6 01:53:02 2022] DELETE /identity/v3/projects/3bea2c3f927a4bccb59065a8380b98c9 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:53:02.406581 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b367e91-eb31-4ac6-b36b-0fac42d588f5 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:53:02.408124 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-2b367e91-eb31-4ac6-b36b-0fac42d588f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:53:02.421594 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b367e91-eb31-4ac6-b36b-0fac42d588f5 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:53:02.433448 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:53:02.480163 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30128adf-1114-4946-bc55-1e197ac3f316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:53:02.480830 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30128adf-1114-4946-bc55-1e197ac3f316 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:53:02.481141 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30128adf-1114-4946-bc55-1e197ac3f316 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:53:02.481530 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30128adf-1114-4946-bc55-1e197ac3f316 admin admin] PATH_INFO: `/v3/users/5083dc0c8e644ad48342790c8c099c7a` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:53:02.482770 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30128adf-1114-4946-bc55-1e197ac3f316 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5083dc0c8e644ad48342790c8c099c7a)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:53:02.484727 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30128adf-1114-4946-bc55-1e197ac3f316 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:53:02.529090 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-30128adf-1114-4946-bc55-1e197ac3f316 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5083dc0c8e644ad48342790c8c099c7a'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:53:02.535937 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-30128adf-1114-4946-bc55-1e197ac3f316 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5083dc0c8e644ad48342790c8c099c7a'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:53:02.536593 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-30128adf-1114-4946-bc55-1e197ac3f316 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5083dc0c8e644ad48342790c8c099c7a'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:53:02.537230 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-30128adf-1114-4946-bc55-1e197ac3f316 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5083dc0c8e644ad48342790c8c099c7a'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:53:02.548799 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-30128adf-1114-4946-bc55-1e197ac3f316 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5083dc0c8e644ad48342790c8c099c7a'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:53:02.561168 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1460/2911] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Aug 6 01:53:02 2022] DELETE /identity/v3/users/5083dc0c8e644ad48342790c8c099c7a => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:53:02.735165 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c1dfea-12a6-403c-9511-3a2c5f6c5ad0 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:53:02.736602 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-62c1dfea-12a6-403c-9511-3a2c5f6c5ad0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:53:02.751553 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c1dfea-12a6-403c-9511-3a2c5f6c5ad0 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:53:02.810127 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66d98ca-e99e-443c-96d9-fdbf07a2b773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca5d3774b93a41d1a8b95796b7899d4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477e329599354f58af86e800a455a8b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:53:02.810772 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66d98ca-e99e-443c-96d9-fdbf07a2b773 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:53:02.811058 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66d98ca-e99e-443c-96d9-fdbf07a2b773 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:53:02.811338 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66d98ca-e99e-443c-96d9-fdbf07a2b773 admin admin] PATH_INFO: `/v3/projects/c9a8221dc2ea41ea8bcb8a829cfc454f` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:53:02.812502 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66d98ca-e99e-443c-96d9-fdbf07a2b773 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9a8221dc2ea41ea8bcb8a829cfc454f)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:53:02.814446 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66d98ca-e99e-443c-96d9-fdbf07a2b773 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:53:02.835180 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:53:02.861846 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-b66d98ca-e99e-443c-96d9-fdbf07a2b773 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9a8221dc2ea41ea8bcb8a829cfc454f', 'request_id': 'req-b66d98ca-e99e-443c-96d9-fdbf07a2b773'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:53:02.867842 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-b66d98ca-e99e-443c-96d9-fdbf07a2b773 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9a8221dc2ea41ea8bcb8a829cfc454f', 'request_id': 'req-b66d98ca-e99e-443c-96d9-fdbf07a2b773'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:53:02.873384 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-b66d98ca-e99e-443c-96d9-fdbf07a2b773 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9a8221dc2ea41ea8bcb8a829cfc454f', 'request_id': 'req-b66d98ca-e99e-443c-96d9-fdbf07a2b773'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:53:02.885331 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-b66d98ca-e99e-443c-96d9-fdbf07a2b773 admin admin] The token cache is being invalidate because project c9a8221dc2ea41ea8bcb8a829cfc454f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:53:02.885623 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-b66d98ca-e99e-443c-96d9-fdbf07a2b773 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:53:02.886245 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-b66d98ca-e99e-443c-96d9-fdbf07a2b773 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:53:02.888012 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1452/2912] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Aug 6 01:53:02 2022] DELETE /identity/v3/projects/c9a8221dc2ea41ea8bcb8a829cfc454f => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:53:12.557476 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=45,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:53:12.882224 dv-cinder-852009-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:57:16.285170 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30128adf-1114-4946-bc55-1e197ac3f316 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:57:16.286598 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-30128adf-1114-4946-bc55-1e197ac3f316 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:57:16.355723 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30128adf-1114-4946-bc55-1e197ac3f316 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:57:16.413777 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48a51d4-762f-41d0-a905-4d2e1711f8b2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa3b93ddd9464a2cba724d67b56b477a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9d68d310bf24449a5fe20d2dc264a58', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:57:16.415005 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48a51d4-762f-41d0-a905-4d2e1711f8b2 service placement] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:57:16.415325 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48a51d4-762f-41d0-a905-4d2e1711f8b2 service placement] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:57:16.415642 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48a51d4-762f-41d0-a905-4d2e1711f8b2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:57:16.417443 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-d48a51d4-762f-41d0-a905-4d2e1711f8b2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:57:16.428468 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48a51d4-762f-41d0-a905-4d2e1711f8b2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:57:16.430301 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48a51d4-762f-41d0-a905-4d2e1711f8b2 service placement] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:57:16.440085 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1461/2913] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Aug 6 01:57:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:57:26.435866 dv-cinder-852009-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=58 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}